Skip to content

feat: desktop quick-ask, parallel indexing, UI/UX fixes & agent rework#1030

Merged
MODSetter merged 135 commits intomainfrom
dev
Mar 29, 2026
Merged

feat: desktop quick-ask, parallel indexing, UI/UX fixes & agent rework#1030
MODSetter merged 135 commits intomainfrom
dev

Conversation

@MODSetter
Copy link
Copy Markdown
Owner

@MODSetter MODSetter commented Mar 29, 2026

PR Title: Weekly dev rollup: desktop quick-ask, parallel indexing, UI/UX fixes & performance improvements

Description:

Combines a week of contributions across the stack:

  • Desktop (Electron): Refactored main process into modular files (errors, server, window, deep-links, auto-updater, menu, ipc); added quick-ask floating panel with global shortcut, cross-platform panel support, and clipboard pre-fill.
  • Backend – Indexing: Migrated Google Drive/Calendar/Gmail, Confluence, Jira, Linear, and Notion indexers to a unified parallel pipeline; added streaming downloads, thread-safe embeddings, batch indexing, and performance logging.
  • Frontend – Features: Folder management (create/delete/move), multi-format document export, session-stored tab state, real-time document-type counts, lazy-loaded syntax highlighter, and loading states for team roles/invites.
  • Frontend – Fixes & Polish: Converted pages to server components, improved accessibility (ARIA roles, focus-visible, keyboard nav), replaced client-side nav with <Link>, cleaned up useEffect/useState anti-patterns, added error toasts, and enhanced global error handling with PostHog reporting.
  • Chore: Linting, .gitignore update, metadata for marketing pages, Python skills config, and minor refactors (icons, animation callbacks, carousel state).

Motivation and Context

FIX #

Screenshots

API Changes

  • This PR includes API changes

Change Type

  • Bug fix
  • New feature
  • Performance improvement
  • Refactoring
  • Documentation
  • Dependency/Build system
  • Breaking change
  • Other (specify):

Testing Performed

  • Tested locally
  • Manual/QA verification

Checklist

  • Follows project coding standards and conventions
  • Documentation updated as needed
  • Dependencies updated as needed
  • No lint/build errors or new warnings
  • All relevant tests are passing

High-level PR Summary

This PR combines a week of contributions across desktop, backend, and frontend. The desktop app now has a modular architecture with a global quick-ask panel featuring clipboard pre-fill. The backend introduces a unified parallel indexing pipeline for all major connectors (Google Drive, Calendar, Gmail, Confluence, Jira, Linear, Notion) with streaming downloads and thread-safe embeddings. A new folder management system with drag-and-drop, move, and reorder capabilities has been added. The agent system was reworked to use pre-search middleware that seeds a scoped filesystem instead of direct knowledge base search tools, and the sandbox execution feature was removed. The frontend gained multi-format document export, session-persisted tabs, real-time document type counts via Zero sync, lazy-loaded syntax highlighting, and improved accessibility throughout. Pages were converted to server components where appropriate, client-side navigation replaced with Link components, and error handling enhanced with PostHog reporting.

⏱️ Estimated Review Time: 3+ hours

💡 Review Order Suggestion
Order File Path
1 surfsense_backend/app/db.py
2 surfsense_backend/alembic/versions/109_add_folders_table.py
3 surfsense_backend/app/schemas/folders.py
4 surfsense_backend/app/routes/folders_routes.py
5 surfsense_backend/app/services/folder_service.py
6 surfsense_web/zero/schema/folders.ts
7 surfsense_web/components/documents/FolderNode.tsx
8 surfsense_web/components/documents/FolderTreeView.tsx
9 surfsense_web/components/documents/DocumentNode.tsx
10 surfsense_web/components/layout/ui/sidebar/DocumentsSidebar.tsx
11 surfsense_web/lib/apis/folders-api.service.ts
12 surfsense_backend/app/indexing_pipeline/indexing_pipeline_service.py
13 surfsense_backend/app/indexing_pipeline/document_hashing.py
14 surfsense_backend/app/retriever/chunks_hybrid_search.py
15 surfsense_backend/app/retriever/documents_hybrid_search.py
16 surfsense_backend/app/tasks/connector_indexers/google_drive_indexer.py
17 surfsense_backend/app/tasks/connector_indexers/confluence_indexer.py
18 surfsense_backend/app/tasks/connector_indexers/jira_indexer.py
19 surfsense_backend/app/tasks/connector_indexers/linear_indexer.py
20 surfsense_backend/app/tasks/connector_indexers/notion_indexer.py
21 surfsense_backend/app/tasks/connector_indexers/google_calendar_indexer.py
22 surfsense_backend/app/tasks/connector_indexers/google_gmail_indexer.py
23 surfsense_backend/app/connectors/google_drive/client.py
24 surfsense_backend/app/connectors/google_drive/content_extractor.py
25 surfsense_backend/app/agents/new_chat/chat_deepagent.py
26 surfsense_backend/app/agents/new_chat/middleware/__init__.py
27 surfsense_backend/app/agents/new_chat/middleware/knowledge_search.py
28 surfsense_backend/app/agents/new_chat/middleware/filesystem.py
29 surfsense_backend/app/agents/new_chat/system_prompt.py
30 surfsense_backend/app/agents/new_chat/tools/knowledge_base.py
31 surfsense_backend/app/tasks/chat/stream_new_chat.py
32 surfsense_desktop/src/main.ts
33 surfsense_desktop/src/modules/window.ts
34 surfsense_desktop/src/modules/server.ts
35 surfsense_desktop/src/modules/deep-links.ts
36 surfsense_desktop/src/modules/quick-ask.ts
37 surfsense_desktop/src/modules/errors.ts
38 surfsense_desktop/src/modules/auto-updater.ts
39 surfsense_desktop/src/modules/menu.ts
40 surfsense_desktop/src/ipc/handlers.ts
41 surfsense_desktop/src/ipc/channels.ts
42 surfsense_desktop/src/preload.ts
43 surfsense_web/components/assistant-ui/thread.tsx
44 surfsense_web/components/assistant-ui/inline-mention-editor.tsx
45 surfsense_web/components/assistant-ui/markdown-text.tsx
46 surfsense_web/components/assistant-ui/markdown-code-block.tsx
47 surfsense_web/atoms/tabs/tabs.atom.ts
48 surfsense_web/components/layout/ui/tabs/TabBar.tsx
49 surfsense_web/components/layout/ui/tabs/DocumentTabContent.tsx
50 surfsense_web/components/layout/ui/shell/LayoutShell.tsx
51 surfsense_web/app/dashboard/[search_space_id]/documents/(manage)/components/DocumentsFilters.tsx
52 surfsense_web/components/shared/ExportMenuItems.tsx
53 surfsense_backend/app/routes/documents_routes.py
54 surfsense_backend/app/routes/editor_routes.py
55 surfsense_web/app/dashboard/[search_space_id]/page.tsx
56 surfsense_web/app/public/[token]/page.tsx
57 surfsense_web/app/error.tsx
58 surfsense_web/app/global-error.tsx
59 surfsense_web/components/layout/providers/LayoutDataProvider.tsx
60 surfsense_web/hooks/use-zero-document-type-counts.ts
61 surfsense_web/app/dashboard/[search_space_id]/new-chat/loading.tsx
62 surfsense_web/app/dashboard/[search_space_id]/logs/loading.tsx
63 surfsense_web/app/dashboard/[search_space_id]/more-pages/loading.tsx
64 .cursor/skills/python-patterns/SKILL.md
65 .cursor/skills/python-patterns/metadata.json
66 .gitignore
67 surfsense_backend/.env.example
68 surfsense_backend/pyproject.toml

Need help? Join our Discord

Analyze latest changes

CREDO23 and others added 30 commits March 20, 2026 19:44
…document migration

- Added `download_and_extract_content` function to extract content from Google Drive files as markdown.
- Updated Google Drive indexer to utilize the new content extraction method.
- Implemented document migration logic to update legacy Composio document types to their native Google types.
- Introduced identifier hashing for stable document identification.
- Improved file pre-filtering to handle unchanged and rename-only files efficiently.
- Introduced integration tests for Calendar, Drive, and Gmail indexers to ensure proper document creation and migration.
- Added tests for batch indexing functionality to validate the processing of multiple documents.
- Implemented tests for legacy document migration to verify updates to document types and hashes.
- Enhanced test coverage for the IndexingPipelineService to ensure robust functionality across various document types.
- Modified the `_should_skip_file` function to prevent skipping of documents with a FAILED status, ensuring they are reprocessed even if their content remains unchanged.
- Added a new integration test to verify that FAILED documents are not skipped during the indexing process.
AnishSarkar22 and others added 27 commits March 27, 2026 23:14
feat: add metadata to changelog, contact, and pricing pages
…-button

fix: add focus-visible styles to share button for improved accessibility
feat: enhance accessibility of ExpandedMediaOverlay
fix: add error toast to invite creation failure
…tting

feat: add error handling for search space loading in GeneralSettingsManager
… and UI components

- Implemented a new export endpoint in the backend to support exporting documents in various formats (PDF, DOCX, HTML, LaTeX, EPUB, ODT, plain text).
- Enhanced DocumentNode and FolderTreeView components to include export options in context and dropdown menus.
- Created shared ExportMenuItems component for consistent export options across the application.
- Integrated loading indicators for export actions to improve user experience.
feat: refactor indexing pipelines for some connectors
Remove useEffect that watched stepIndex to drive shouldAnimate/contentKey.
stepIndex only changes from handleNext/handlePrev, so set shouldAnimate
directly in those handlers. Replace contentKey state (always equal to
stepIndex) with stepIndex as the animation key.

Fixes #1017
- Updated DocumentNode to use a div instead of a button for better accessibility and added keyboard interaction for selection.
- Introduced Checkbox component in FolderNode for selecting folders, with state management for selection (all, some, none).
- Implemented folder selection state logic in FolderTreeView to manage document selection across nested folders.
- Added handleToggleFolderSelect function in DocumentsSidebar to manage selection of documents based on folder selection.
Switched goTo, goToPrev, and goToNext from depending on activeIndex
to using functional setState updates. Removed activeIndex from the
autoplay effect dependency array since the timer callback already
uses a functional update. Callbacks are now stable references.

Fixes #951
…-invites

feat: add loading states for roles and invites in TeamContent component
Add aria-label and aria-pressed to web search toggle
…-in-handlers

refactor: move onboarding animation state into event handlers
Lazy-load syntax highlighter in markdown text renderer
refactor: use functional state updates in hero carousel callbacks
@vercel
Copy link
Copy Markdown

vercel bot commented Mar 29, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
surf-sense-frontend Ready Ready Preview, Comment Mar 29, 2026 9:56am

Request Review

@MODSetter MODSetter merged commit c74b517 into main Mar 29, 2026
13 of 17 checks passed
Copy link
Copy Markdown

@recurseml recurseml bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review by RecurseML

🔍 Review performed on dfe3e7b..cae4262

✨ No bugs found, your code is sparkling clean

✅ Files analyzed, no issues (50)

.cursor/skills/python-patterns/SKILL.md
.cursor/skills/python-patterns/metadata.json
.gitignore
surfsense_backend/.env.example
surfsense_backend/alembic/versions/107_add_video_presentations_table.py
surfsense_backend/alembic/versions/109_add_folders_table.py
surfsense_backend/app/agents/new_chat/__init__.py
surfsense_backend/app/agents/new_chat/chat_deepagent.py
surfsense_backend/app/agents/new_chat/middleware/__init__.py
surfsense_backend/app/agents/new_chat/middleware/filesystem.py
surfsense_backend/app/agents/new_chat/middleware/knowledge_search.py
surfsense_backend/app/agents/new_chat/system_prompt.py
surfsense_backend/app/agents/new_chat/tools/__init__.py
surfsense_backend/app/agents/new_chat/tools/google_calendar/update_event.py
surfsense_backend/app/agents/new_chat/tools/knowledge_base.py
surfsense_backend/app/agents/new_chat/tools/registry.py
surfsense_backend/app/agents/new_chat/tools/report.py
surfsense_backend/app/connectors/google_drive/__init__.py
surfsense_backend/app/connectors/google_drive/change_tracker.py
surfsense_backend/app/connectors/google_drive/client.py
surfsense_backend/app/connectors/google_drive/content_extractor.py
surfsense_backend/app/connectors/google_drive/file_types.py
surfsense_backend/app/db.py
surfsense_backend/app/indexing_pipeline/document_hashing.py
surfsense_backend/app/indexing_pipeline/indexing_pipeline_service.py
surfsense_backend/app/retriever/chunks_hybrid_search.py
surfsense_backend/app/retriever/documents_hybrid_search.py
surfsense_backend/app/routes/__init__.py
surfsense_backend/app/routes/documents_routes.py
surfsense_backend/app/routes/editor_routes.py
surfsense_backend/app/routes/folders_routes.py
surfsense_backend/app/routes/search_source_connectors_routes.py
surfsense_backend/app/schemas/__init__.py
surfsense_backend/app/schemas/documents.py
surfsense_backend/app/schemas/folders.py
surfsense_backend/app/services/folder_service.py
surfsense_backend/app/services/google_calendar/kb_sync_service.py
surfsense_backend/app/tasks/celery_tasks/document_tasks.py
surfsense_backend/app/tasks/chat/stream_new_chat.py
surfsense_backend/app/tasks/connector_indexers/confluence_indexer.py
surfsense_backend/app/tasks/connector_indexers/google_calendar_indexer.py
surfsense_backend/app/tasks/connector_indexers/google_drive_indexer.py
surfsense_backend/app/tasks/connector_indexers/google_gmail_indexer.py
surfsense_backend/app/tasks/connector_indexers/jira_indexer.py
surfsense_backend/app/tasks/connector_indexers/linear_indexer.py
surfsense_backend/app/tasks/connector_indexers/notion_indexer.py
surfsense_backend/app/utils/document_converters.py
surfsense_backend/pyproject.toml
surfsense_backend/tests/integration/indexing_pipeline/test_calendar_pipeline.py
surfsense_backend/tests/integration/indexing_pipeline/test_drive_pipeline.py

⏭️ Files skipped (128)
  Locations  
surfsense_backend/tests/integration/indexing_pipeline/test_gmail_pipeline.py
surfsense_backend/tests/integration/indexing_pipeline/test_index_batch.py
surfsense_backend/tests/integration/indexing_pipeline/test_migrate_legacy_docs.py
surfsense_backend/tests/integration/retriever/__init__.py
surfsense_backend/tests/integration/retriever/conftest.py
surfsense_backend/tests/integration/retriever/test_optimized_chunk_retriever.py
surfsense_backend/tests/integration/retriever/test_optimized_doc_retriever.py
surfsense_backend/tests/unit/connector_indexers/__init__.py
surfsense_backend/tests/unit/connector_indexers/conftest.py
surfsense_backend/tests/unit/connector_indexers/test_confluence_parallel.py
surfsense_backend/tests/unit/connector_indexers/test_google_drive_parallel.py
surfsense_backend/tests/unit/connector_indexers/test_jira_parallel.py
surfsense_backend/tests/unit/connector_indexers/test_linear_parallel.py
surfsense_backend/tests/unit/connector_indexers/test_notion_parallel.py
surfsense_backend/tests/unit/indexing_pipeline/test_create_placeholder_documents.py
surfsense_backend/tests/unit/indexing_pipeline/test_document_hashing.py
surfsense_backend/tests/unit/indexing_pipeline/test_index_batch.py
surfsense_backend/tests/unit/indexing_pipeline/test_index_batch_parallel.py
surfsense_backend/tests/unit/indexing_pipeline/test_migrate_legacy_docs.py
surfsense_backend/tests/unit/indexing_pipeline/test_prepare_placeholder_dedup.py
surfsense_backend/tests/unit/middleware/test_knowledge_search.py
surfsense_backend/uv.lock
surfsense_desktop/src/ipc/channels.ts
surfsense_desktop/src/ipc/handlers.ts
surfsense_desktop/src/main.ts
surfsense_desktop/src/modules/auto-updater.ts
surfsense_desktop/src/modules/deep-links.ts
surfsense_desktop/src/modules/errors.ts
surfsense_desktop/src/modules/menu.ts
surfsense_desktop/src/modules/quick-ask.ts
surfsense_desktop/src/modules/server.ts
surfsense_desktop/src/modules/window.ts
surfsense_desktop/src/preload.ts
surfsense_web/app/(home)/changelog/page.tsx
surfsense_web/app/(home)/contact/page.tsx
surfsense_web/app/(home)/login/LocalLoginForm.tsx
surfsense_web/app/(home)/pricing/page.tsx
surfsense_web/app/(home)/register/page.tsx
surfsense_web/app/dashboard/[search_space_id]/documents/(manage)/components/DocumentTypeIcon.tsx
surfsense_web/app/dashboard/[search_space_id]/documents/(manage)/components/DocumentsFilters.tsx
surfsense_web/app/dashboard/[search_space_id]/documents/(manage)/components/DocumentsTableShell.tsx
surfsense_web/app/dashboard/[search_space_id]/documents/(manage)/components/RowActions.tsx
surfsense_web/app/dashboard/[search_space_id]/logs/loading.tsx
surfsense_web/app/dashboard/[search_space_id]/more-pages/loading.tsx
surfsense_web/app/dashboard/[search_space_id]/new-chat/[[...chat_id]]/page.tsx
surfsense_web/app/dashboard/[search_space_id]/new-chat/loading.tsx
surfsense_web/app/dashboard/[search_space_id]/page.tsx
surfsense_web/app/dashboard/[search_space_id]/team/team-content.tsx
surfsense_web/app/error.tsx
surfsense_web/app/global-error.tsx
surfsense_web/app/public/[token]/page.tsx
surfsense_web/atoms/documents/document-mutation.atoms.ts
surfsense_web/atoms/documents/document-query.atoms.ts
surfsense_web/atoms/documents/folder.atoms.ts
surfsense_web/atoms/documents/ui.atoms.ts
surfsense_web/atoms/tabs/tabs.atom.ts
surfsense_web/components/Logo.tsx
surfsense_web/components/UserDropdown.tsx
surfsense_web/components/announcements/AnnouncementToastProvider.tsx
surfsense_web/components/announcements/AnnouncementsEmptyState.tsx
surfsense_web/components/assistant-ui/connector-popup.tsx
surfsense_web/components/assistant-ui/connector-popup/hooks/use-connector-dialog.ts
surfsense_web/components/assistant-ui/inline-citation.tsx
surfsense_web/components/assistant-ui/inline-mention-editor.tsx
surfsense_web/components/assistant-ui/markdown-code-block.tsx
surfsense_web/components/assistant-ui/markdown-text.tsx
surfsense_web/components/assistant-ui/thread-list.tsx
surfsense_web/components/assistant-ui/thread.tsx
surfsense_web/components/auth/sign-in-button.tsx
surfsense_web/components/contact/contact-form.tsx
surfsense_web/components/documents/CreateFolderDialog.tsx
surfsense_web/components/documents/DocumentNode.tsx
surfsense_web/components/documents/FolderNode.tsx
surfsense_web/components/documents/FolderPickerDialog.tsx
surfsense_web/components/documents/FolderTreeView.tsx
surfsense_web/components/hitl-edit-panel/hitl-edit-panel.tsx
surfsense_web/components/homepage/github-stars-badge.tsx
surfsense_web/components/homepage/hero-section.tsx
surfsense_web/components/homepage/use-cases-grid.tsx
surfsense_web/components/layout/providers/LayoutDataProvider.tsx
surfsense_web/components/layout/ui/header/Header.tsx
surfsense_web/components/layout/ui/shell/LayoutShell.tsx
surfsense_web/components/layout/ui/sidebar/AllPrivateChatsSidebar.tsx
surfsense_web/components/layout/ui/sidebar/AllSharedChatsSidebar.tsx
surfsense_web/components/layout/ui/sidebar/ChatListItem.tsx
surfsense_web/components/layout/ui/sidebar/DocumentsSidebar.tsx
surfsense_web/components/layout/ui/tabs/DocumentTabContent.tsx
surfsense_web/components/layout/ui/tabs/TabBar.tsx
surfsense_web/components/new-chat/chat-share-button.tsx
surfsense_web/components/onboarding-tour.tsx
surfsense_web/components/public-chat-snapshots/public-chat-snapshots-empty-state.tsx
surfsense_web/components/report-panel/report-panel.tsx
surfsense_web/components/settings/general-settings-manager.tsx
surfsense_web/components/settings/prompt-config-manager.tsx
surfsense_web/components/shared/ExportMenuItems.tsx
surfsense_web/components/theme/theme-toggle.tsx
surfsense_web/components/tool-ui/google-calendar/update-event.tsx
surfsense_web/components/tool-ui/sandbox-execute.tsx
surfsense_web/components/ui/block-draggable.tsx
surfsense_web/components/ui/equation-node.tsx
surfsense_web/components/ui/expanded-gif-overlay.tsx
surfsense_web/components/ui/hero-carousel.tsx
surfsense_web/components/ui/link-toolbar.tsx
surfsense_web/components/ui/toolbar.tsx
surfsense_web/content/docs/how-to/zero-sync.mdx
surfsense_web/contracts/enums/toolIcons.tsx
surfsense_web/contracts/types/folder.types.ts
surfsense_web/hooks/use-logs.ts
surfsense_web/hooks/use-search-source-connectors.ts
surfsense_web/hooks/use-zero-document-type-counts.ts
surfsense_web/i18n/request.ts
surfsense_web/lib/apis/base-api.service.ts
surfsense_web/lib/apis/documents-api.service.ts
surfsense_web/lib/apis/folders-api.service.ts
surfsense_web/lib/apis/invites-api.service.ts
surfsense_web/lib/apis/logs-api.service.ts
surfsense_web/lib/apis/members-api.service.ts
surfsense_web/lib/chat/streaming-state.ts
surfsense_web/lib/query-client/cache-keys.ts
surfsense_web/next.config.ts
surfsense_web/package.json
surfsense_web/pnpm-lock.yaml
surfsense_web/types/window.d.ts
surfsense_web/zero/queries/folders.ts
surfsense_web/zero/queries/index.ts
surfsense_web/zero/schema/documents.ts
surfsense_web/zero/schema/folders.ts
surfsense_web/zero/schema/index.ts

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants