Conversation
…document migration - Added `download_and_extract_content` function to extract content from Google Drive files as markdown. - Updated Google Drive indexer to utilize the new content extraction method. - Implemented document migration logic to update legacy Composio document types to their native Google types. - Introduced identifier hashing for stable document identification. - Improved file pre-filtering to handle unchanged and rename-only files efficiently.
- Introduced integration tests for Calendar, Drive, and Gmail indexers to ensure proper document creation and migration. - Added tests for batch indexing functionality to validate the processing of multiple documents. - Implemented tests for legacy document migration to verify updates to document types and hashes. - Enhanced test coverage for the IndexingPipelineService to ensure robust functionality across various document types.
- Modified the `_should_skip_file` function to prevent skipping of documents with a FAILED status, ensuring they are reprocessed even if their content remains unchanged. - Added a new integration test to verify that FAILED documents are not skipped during the indexing process.
…clePlus in document components
feat: add metadata to changelog, contact, and pricing pages
…-button fix: add focus-visible styles to share button for improved accessibility
feat: enhance accessibility of ExpandedMediaOverlay
fix: add error toast to invite creation failure
…tting feat: add error handling for search space loading in GeneralSettingsManager
… and UI components - Implemented a new export endpoint in the backend to support exporting documents in various formats (PDF, DOCX, HTML, LaTeX, EPUB, ODT, plain text). - Enhanced DocumentNode and FolderTreeView components to include export options in context and dropdown menus. - Created shared ExportMenuItems component for consistent export options across the application. - Integrated loading indicators for export actions to improve user experience.
feat: refactor indexing pipelines for some connectors
Remove useEffect that watched stepIndex to drive shouldAnimate/contentKey. stepIndex only changes from handleNext/handlePrev, so set shouldAnimate directly in those handlers. Replace contentKey state (always equal to stepIndex) with stepIndex as the animation key. Fixes #1017
- Updated DocumentNode to use a div instead of a button for better accessibility and added keyboard interaction for selection. - Introduced Checkbox component in FolderNode for selecting folders, with state management for selection (all, some, none). - Implemented folder selection state logic in FolderTreeView to manage document selection across nested folders. - Added handleToggleFolderSelect function in DocumentsSidebar to manage selection of documents based on folder selection.
Switched goTo, goToPrev, and goToNext from depending on activeIndex to using functional setState updates. Removed activeIndex from the autoplay effect dependency array since the timer callback already uses a functional update. Callbacks are now stable references. Fixes #951
feat: new file based agent
…-invites feat: add loading states for roles and invites in TeamContent component
Add aria-label and aria-pressed to web search toggle
…-in-handlers refactor: move onboarding animation state into event handlers
Lazy-load syntax highlighter in markdown text renderer
refactor: use functional state updates in hero carousel callbacks
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
There was a problem hiding this comment.
Review by RecurseML
🔍 Review performed on dfe3e7b..cae4262
✨ No bugs found, your code is sparkling clean
✅ Files analyzed, no issues (50)
• .cursor/skills/python-patterns/SKILL.md
• .cursor/skills/python-patterns/metadata.json
• .gitignore
• surfsense_backend/.env.example
• surfsense_backend/alembic/versions/107_add_video_presentations_table.py
• surfsense_backend/alembic/versions/109_add_folders_table.py
• surfsense_backend/app/agents/new_chat/__init__.py
• surfsense_backend/app/agents/new_chat/chat_deepagent.py
• surfsense_backend/app/agents/new_chat/middleware/__init__.py
• surfsense_backend/app/agents/new_chat/middleware/filesystem.py
• surfsense_backend/app/agents/new_chat/middleware/knowledge_search.py
• surfsense_backend/app/agents/new_chat/system_prompt.py
• surfsense_backend/app/agents/new_chat/tools/__init__.py
• surfsense_backend/app/agents/new_chat/tools/google_calendar/update_event.py
• surfsense_backend/app/agents/new_chat/tools/knowledge_base.py
• surfsense_backend/app/agents/new_chat/tools/registry.py
• surfsense_backend/app/agents/new_chat/tools/report.py
• surfsense_backend/app/connectors/google_drive/__init__.py
• surfsense_backend/app/connectors/google_drive/change_tracker.py
• surfsense_backend/app/connectors/google_drive/client.py
• surfsense_backend/app/connectors/google_drive/content_extractor.py
• surfsense_backend/app/connectors/google_drive/file_types.py
• surfsense_backend/app/db.py
• surfsense_backend/app/indexing_pipeline/document_hashing.py
• surfsense_backend/app/indexing_pipeline/indexing_pipeline_service.py
• surfsense_backend/app/retriever/chunks_hybrid_search.py
• surfsense_backend/app/retriever/documents_hybrid_search.py
• surfsense_backend/app/routes/__init__.py
• surfsense_backend/app/routes/documents_routes.py
• surfsense_backend/app/routes/editor_routes.py
• surfsense_backend/app/routes/folders_routes.py
• surfsense_backend/app/routes/search_source_connectors_routes.py
• surfsense_backend/app/schemas/__init__.py
• surfsense_backend/app/schemas/documents.py
• surfsense_backend/app/schemas/folders.py
• surfsense_backend/app/services/folder_service.py
• surfsense_backend/app/services/google_calendar/kb_sync_service.py
• surfsense_backend/app/tasks/celery_tasks/document_tasks.py
• surfsense_backend/app/tasks/chat/stream_new_chat.py
• surfsense_backend/app/tasks/connector_indexers/confluence_indexer.py
• surfsense_backend/app/tasks/connector_indexers/google_calendar_indexer.py
• surfsense_backend/app/tasks/connector_indexers/google_drive_indexer.py
• surfsense_backend/app/tasks/connector_indexers/google_gmail_indexer.py
• surfsense_backend/app/tasks/connector_indexers/jira_indexer.py
• surfsense_backend/app/tasks/connector_indexers/linear_indexer.py
• surfsense_backend/app/tasks/connector_indexers/notion_indexer.py
• surfsense_backend/app/utils/document_converters.py
• surfsense_backend/pyproject.toml
• surfsense_backend/tests/integration/indexing_pipeline/test_calendar_pipeline.py
• surfsense_backend/tests/integration/indexing_pipeline/test_drive_pipeline.py
⏭️ Files skipped (128)
| Locations |
|---|
surfsense_backend/tests/integration/indexing_pipeline/test_gmail_pipeline.py |
surfsense_backend/tests/integration/indexing_pipeline/test_index_batch.py |
surfsense_backend/tests/integration/indexing_pipeline/test_migrate_legacy_docs.py |
surfsense_backend/tests/integration/retriever/__init__.py |
surfsense_backend/tests/integration/retriever/conftest.py |
surfsense_backend/tests/integration/retriever/test_optimized_chunk_retriever.py |
surfsense_backend/tests/integration/retriever/test_optimized_doc_retriever.py |
surfsense_backend/tests/unit/connector_indexers/__init__.py |
surfsense_backend/tests/unit/connector_indexers/conftest.py |
surfsense_backend/tests/unit/connector_indexers/test_confluence_parallel.py |
surfsense_backend/tests/unit/connector_indexers/test_google_drive_parallel.py |
surfsense_backend/tests/unit/connector_indexers/test_jira_parallel.py |
surfsense_backend/tests/unit/connector_indexers/test_linear_parallel.py |
surfsense_backend/tests/unit/connector_indexers/test_notion_parallel.py |
surfsense_backend/tests/unit/indexing_pipeline/test_create_placeholder_documents.py |
surfsense_backend/tests/unit/indexing_pipeline/test_document_hashing.py |
surfsense_backend/tests/unit/indexing_pipeline/test_index_batch.py |
surfsense_backend/tests/unit/indexing_pipeline/test_index_batch_parallel.py |
surfsense_backend/tests/unit/indexing_pipeline/test_migrate_legacy_docs.py |
surfsense_backend/tests/unit/indexing_pipeline/test_prepare_placeholder_dedup.py |
surfsense_backend/tests/unit/middleware/test_knowledge_search.py |
surfsense_backend/uv.lock |
surfsense_desktop/src/ipc/channels.ts |
surfsense_desktop/src/ipc/handlers.ts |
surfsense_desktop/src/main.ts |
surfsense_desktop/src/modules/auto-updater.ts |
surfsense_desktop/src/modules/deep-links.ts |
surfsense_desktop/src/modules/errors.ts |
surfsense_desktop/src/modules/menu.ts |
surfsense_desktop/src/modules/quick-ask.ts |
surfsense_desktop/src/modules/server.ts |
surfsense_desktop/src/modules/window.ts |
surfsense_desktop/src/preload.ts |
surfsense_web/app/(home)/changelog/page.tsx |
surfsense_web/app/(home)/contact/page.tsx |
surfsense_web/app/(home)/login/LocalLoginForm.tsx |
surfsense_web/app/(home)/pricing/page.tsx |
surfsense_web/app/(home)/register/page.tsx |
surfsense_web/app/dashboard/[search_space_id]/documents/(manage)/components/DocumentTypeIcon.tsx |
surfsense_web/app/dashboard/[search_space_id]/documents/(manage)/components/DocumentsFilters.tsx |
surfsense_web/app/dashboard/[search_space_id]/documents/(manage)/components/DocumentsTableShell.tsx |
surfsense_web/app/dashboard/[search_space_id]/documents/(manage)/components/RowActions.tsx |
surfsense_web/app/dashboard/[search_space_id]/logs/loading.tsx |
surfsense_web/app/dashboard/[search_space_id]/more-pages/loading.tsx |
surfsense_web/app/dashboard/[search_space_id]/new-chat/[[...chat_id]]/page.tsx |
surfsense_web/app/dashboard/[search_space_id]/new-chat/loading.tsx |
surfsense_web/app/dashboard/[search_space_id]/page.tsx |
surfsense_web/app/dashboard/[search_space_id]/team/team-content.tsx |
surfsense_web/app/error.tsx |
surfsense_web/app/global-error.tsx |
surfsense_web/app/public/[token]/page.tsx |
surfsense_web/atoms/documents/document-mutation.atoms.ts |
surfsense_web/atoms/documents/document-query.atoms.ts |
surfsense_web/atoms/documents/folder.atoms.ts |
surfsense_web/atoms/documents/ui.atoms.ts |
surfsense_web/atoms/tabs/tabs.atom.ts |
surfsense_web/components/Logo.tsx |
surfsense_web/components/UserDropdown.tsx |
surfsense_web/components/announcements/AnnouncementToastProvider.tsx |
surfsense_web/components/announcements/AnnouncementsEmptyState.tsx |
surfsense_web/components/assistant-ui/connector-popup.tsx |
surfsense_web/components/assistant-ui/connector-popup/hooks/use-connector-dialog.ts |
surfsense_web/components/assistant-ui/inline-citation.tsx |
surfsense_web/components/assistant-ui/inline-mention-editor.tsx |
surfsense_web/components/assistant-ui/markdown-code-block.tsx |
surfsense_web/components/assistant-ui/markdown-text.tsx |
surfsense_web/components/assistant-ui/thread-list.tsx |
surfsense_web/components/assistant-ui/thread.tsx |
surfsense_web/components/auth/sign-in-button.tsx |
surfsense_web/components/contact/contact-form.tsx |
surfsense_web/components/documents/CreateFolderDialog.tsx |
surfsense_web/components/documents/DocumentNode.tsx |
surfsense_web/components/documents/FolderNode.tsx |
surfsense_web/components/documents/FolderPickerDialog.tsx |
surfsense_web/components/documents/FolderTreeView.tsx |
surfsense_web/components/hitl-edit-panel/hitl-edit-panel.tsx |
surfsense_web/components/homepage/github-stars-badge.tsx |
surfsense_web/components/homepage/hero-section.tsx |
surfsense_web/components/homepage/use-cases-grid.tsx |
surfsense_web/components/layout/providers/LayoutDataProvider.tsx |
surfsense_web/components/layout/ui/header/Header.tsx |
surfsense_web/components/layout/ui/shell/LayoutShell.tsx |
surfsense_web/components/layout/ui/sidebar/AllPrivateChatsSidebar.tsx |
surfsense_web/components/layout/ui/sidebar/AllSharedChatsSidebar.tsx |
surfsense_web/components/layout/ui/sidebar/ChatListItem.tsx |
surfsense_web/components/layout/ui/sidebar/DocumentsSidebar.tsx |
surfsense_web/components/layout/ui/tabs/DocumentTabContent.tsx |
surfsense_web/components/layout/ui/tabs/TabBar.tsx |
surfsense_web/components/new-chat/chat-share-button.tsx |
surfsense_web/components/onboarding-tour.tsx |
surfsense_web/components/public-chat-snapshots/public-chat-snapshots-empty-state.tsx |
surfsense_web/components/report-panel/report-panel.tsx |
surfsense_web/components/settings/general-settings-manager.tsx |
surfsense_web/components/settings/prompt-config-manager.tsx |
surfsense_web/components/shared/ExportMenuItems.tsx |
surfsense_web/components/theme/theme-toggle.tsx |
surfsense_web/components/tool-ui/google-calendar/update-event.tsx |
surfsense_web/components/tool-ui/sandbox-execute.tsx |
surfsense_web/components/ui/block-draggable.tsx |
surfsense_web/components/ui/equation-node.tsx |
surfsense_web/components/ui/expanded-gif-overlay.tsx |
surfsense_web/components/ui/hero-carousel.tsx |
surfsense_web/components/ui/link-toolbar.tsx |
surfsense_web/components/ui/toolbar.tsx |
surfsense_web/content/docs/how-to/zero-sync.mdx |
surfsense_web/contracts/enums/toolIcons.tsx |
surfsense_web/contracts/types/folder.types.ts |
surfsense_web/hooks/use-logs.ts |
surfsense_web/hooks/use-search-source-connectors.ts |
surfsense_web/hooks/use-zero-document-type-counts.ts |
surfsense_web/i18n/request.ts |
surfsense_web/lib/apis/base-api.service.ts |
surfsense_web/lib/apis/documents-api.service.ts |
surfsense_web/lib/apis/folders-api.service.ts |
surfsense_web/lib/apis/invites-api.service.ts |
surfsense_web/lib/apis/logs-api.service.ts |
surfsense_web/lib/apis/members-api.service.ts |
surfsense_web/lib/chat/streaming-state.ts |
surfsense_web/lib/query-client/cache-keys.ts |
surfsense_web/next.config.ts |
surfsense_web/package.json |
surfsense_web/pnpm-lock.yaml |
surfsense_web/types/window.d.ts |
surfsense_web/zero/queries/folders.ts |
surfsense_web/zero/queries/index.ts |
surfsense_web/zero/schema/documents.ts |
surfsense_web/zero/schema/folders.ts |
surfsense_web/zero/schema/index.ts |
PR Title: Weekly dev rollup: desktop quick-ask, parallel indexing, UI/UX fixes & performance improvements
Description:
Motivation and Context
FIX #
Screenshots
API Changes
Change Type
Testing Performed
Checklist
High-level PR Summary
This PR combines a week of contributions across desktop, backend, and frontend. The desktop app now has a modular architecture with a global quick-ask panel featuring clipboard pre-fill. The backend introduces a unified parallel indexing pipeline for all major connectors (Google Drive, Calendar, Gmail, Confluence, Jira, Linear, Notion) with streaming downloads and thread-safe embeddings. A new folder management system with drag-and-drop, move, and reorder capabilities has been added. The agent system was reworked to use pre-search middleware that seeds a scoped filesystem instead of direct knowledge base search tools, and the sandbox execution feature was removed. The frontend gained multi-format document export, session-persisted tabs, real-time document type counts via Zero sync, lazy-loaded syntax highlighting, and improved accessibility throughout. Pages were converted to server components where appropriate, client-side navigation replaced with
Linkcomponents, and error handling enhanced with PostHog reporting.⏱️ Estimated Review Time: 3+ hours
💡 Review Order Suggestion
surfsense_backend/app/db.pysurfsense_backend/alembic/versions/109_add_folders_table.pysurfsense_backend/app/schemas/folders.pysurfsense_backend/app/routes/folders_routes.pysurfsense_backend/app/services/folder_service.pysurfsense_web/zero/schema/folders.tssurfsense_web/components/documents/FolderNode.tsxsurfsense_web/components/documents/FolderTreeView.tsxsurfsense_web/components/documents/DocumentNode.tsxsurfsense_web/components/layout/ui/sidebar/DocumentsSidebar.tsxsurfsense_web/lib/apis/folders-api.service.tssurfsense_backend/app/indexing_pipeline/indexing_pipeline_service.pysurfsense_backend/app/indexing_pipeline/document_hashing.pysurfsense_backend/app/retriever/chunks_hybrid_search.pysurfsense_backend/app/retriever/documents_hybrid_search.pysurfsense_backend/app/tasks/connector_indexers/google_drive_indexer.pysurfsense_backend/app/tasks/connector_indexers/confluence_indexer.pysurfsense_backend/app/tasks/connector_indexers/jira_indexer.pysurfsense_backend/app/tasks/connector_indexers/linear_indexer.pysurfsense_backend/app/tasks/connector_indexers/notion_indexer.pysurfsense_backend/app/tasks/connector_indexers/google_calendar_indexer.pysurfsense_backend/app/tasks/connector_indexers/google_gmail_indexer.pysurfsense_backend/app/connectors/google_drive/client.pysurfsense_backend/app/connectors/google_drive/content_extractor.pysurfsense_backend/app/agents/new_chat/chat_deepagent.pysurfsense_backend/app/agents/new_chat/middleware/__init__.pysurfsense_backend/app/agents/new_chat/middleware/knowledge_search.pysurfsense_backend/app/agents/new_chat/middleware/filesystem.pysurfsense_backend/app/agents/new_chat/system_prompt.pysurfsense_backend/app/agents/new_chat/tools/knowledge_base.pysurfsense_backend/app/tasks/chat/stream_new_chat.pysurfsense_desktop/src/main.tssurfsense_desktop/src/modules/window.tssurfsense_desktop/src/modules/server.tssurfsense_desktop/src/modules/deep-links.tssurfsense_desktop/src/modules/quick-ask.tssurfsense_desktop/src/modules/errors.tssurfsense_desktop/src/modules/auto-updater.tssurfsense_desktop/src/modules/menu.tssurfsense_desktop/src/ipc/handlers.tssurfsense_desktop/src/ipc/channels.tssurfsense_desktop/src/preload.tssurfsense_web/components/assistant-ui/thread.tsxsurfsense_web/components/assistant-ui/inline-mention-editor.tsxsurfsense_web/components/assistant-ui/markdown-text.tsxsurfsense_web/components/assistant-ui/markdown-code-block.tsxsurfsense_web/atoms/tabs/tabs.atom.tssurfsense_web/components/layout/ui/tabs/TabBar.tsxsurfsense_web/components/layout/ui/tabs/DocumentTabContent.tsxsurfsense_web/components/layout/ui/shell/LayoutShell.tsxsurfsense_web/app/dashboard/[search_space_id]/documents/(manage)/components/DocumentsFilters.tsxsurfsense_web/components/shared/ExportMenuItems.tsxsurfsense_backend/app/routes/documents_routes.pysurfsense_backend/app/routes/editor_routes.pysurfsense_web/app/dashboard/[search_space_id]/page.tsxsurfsense_web/app/public/[token]/page.tsxsurfsense_web/app/error.tsxsurfsense_web/app/global-error.tsxsurfsense_web/components/layout/providers/LayoutDataProvider.tsxsurfsense_web/hooks/use-zero-document-type-counts.tssurfsense_web/app/dashboard/[search_space_id]/new-chat/loading.tsxsurfsense_web/app/dashboard/[search_space_id]/logs/loading.tsxsurfsense_web/app/dashboard/[search_space_id]/more-pages/loading.tsx.cursor/skills/python-patterns/SKILL.md.cursor/skills/python-patterns/metadata.json.gitignoresurfsense_backend/.env.examplesurfsense_backend/pyproject.toml