Skip to content

Feature/paperless-ngx connector#4609

Closed
cbrown350 wants to merge 23 commits intoonyx-dot-app:mainfrom
cbrown350:Feature/Paperless-ngx_Connector
Closed

Feature/paperless-ngx connector#4609
cbrown350 wants to merge 23 commits intoonyx-dot-app:mainfrom
cbrown350:Feature/Paperless-ngx_Connector

Conversation

@cbrown350
Copy link
Copy Markdown

@cbrown350 cbrown350 commented Apr 25, 2025

Description

This adds code for a Paperless-ngx connector: https://docs.paperless-ngx.com

How Has This Been Tested?

This includes the standard mock testing similar to other connectors. Additionally, it has been manually tested extensively. Anyone can easily test it using a clean Docker Paperless-ngx image by running the included script at backend\tests\daily\connectors\paperless_ngx\run_manual_test_paperless_server.sh. This script sets up a server and then provides credentials you can use to create a connector in a local Onyx instance.

Backporting (check the box to trigger backport action)

Note: You have to check that the action passes, otherwise resolve the conflicts manually and tag the patches.

  • This PR should be backported (make sure to check that the backport attempt succeeds)
  • [Optional] Override Linear Check

#159
#776
#3107

onyx-dot-app/documentation#196

@cbrown350 cbrown350 requested a review from a team as a code owner April 25, 2025 05:39
@vercel
Copy link
Copy Markdown

vercel Bot commented Apr 25, 2025

@cbrown350 is attempting to deploy a commit to the Danswer Team on Vercel.

A member of the Team first needs to authorize it.

Copy link
Copy Markdown
Contributor

@greptile-apps greptile-apps Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Summary

This PR adds a new Paperless-ngx connector implementation with comprehensive testing and documentation. Here are the key points:

  • Implements LoadConnector, PollConnector, and SlimConnector interfaces with proper date filtering, tag/user filtering, and error handling in /backend/onyx/connectors/paperless_ngx/connector.py

  • Includes a robust testing setup with both unit tests and a convenient Docker-based manual testing script (run_manual_test_paperless_server.sh) for local validation

  • Adds proper UI configuration in web/src/lib/connectors/connectors.tsx with fields for tag filtering, username filtering, and no-owner document inclusion

  • Bug in date field validation - the code raises an exception when ui_date_field matches a valid field instead of setting it, and lacks proper handling of master vs local date fields

  • Missing error handling for API response validation in _parse_document() - should add checks for required fields and data type validation

💡 (1/5) You can manually trigger the bot by mentioning @greptileai in a comment!

12 file(s) reviewed, 10 comment(s)
Edit PR Review Bot Settings | Greptile

Comment thread backend/tests/daily/connectors/paperless_ngx/run_manual_test_paperless_server.sh Outdated
Comment thread backend/onyx/connectors/paperless_ngx/connector.py Outdated
Comment thread backend/onyx/connectors/paperless_ngx/connector.py Outdated
Comment thread backend/onyx/connectors/paperless_ngx/connector.py
Comment thread backend/onyx/connectors/paperless_ngx/connector.py
Comment thread backend/onyx/connectors/paperless_ngx/connector.py
Comment thread web/src/lib/connectors/credentials.ts
@cbrown350 cbrown350 force-pushed the Feature/Paperless-ngx_Connector branch 2 times, most recently from 4a8b530 to 28bab42 Compare May 1, 2025 05:34
@cbrown350 cbrown350 force-pushed the Feature/Paperless-ngx_Connector branch 2 times, most recently from 8178ad2 to a645789 Compare May 4, 2025 21:22
@cbrown350
Copy link
Copy Markdown
Author

Hello @yuhongsun96, do you know if there's still interest in this connector?

@cbrown350 cbrown350 force-pushed the Feature/Paperless-ngx_Connector branch from a645789 to d20cc20 Compare May 5, 2025 00:51
@github-actions
Copy link
Copy Markdown
Contributor

This PR is stale because it has been open 75 days with no activity. Remove stale label or comment or this will be closed in 15 days.

@github-actions github-actions Bot added the Stale label Jul 19, 2025
@github-actions
Copy link
Copy Markdown
Contributor

This PR was closed because it has been stalled for 90 days with no activity.

@github-actions github-actions Bot closed this Jul 26, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant