Skip to content

Add support to specify custom table schema#138

Open
Anupkumar Ghogare (anupkg1985) wants to merge 4 commits intolangchain-ai:mainfrom
anupkg1985:main
Open

Add support to specify custom table schema#138
Anupkumar Ghogare (anupkg1985) wants to merge 4 commits intolangchain-ai:mainfrom
anupkg1985:main

Conversation

@anupkg1985
Copy link
Copy Markdown

@anupkg1985 Anupkumar Ghogare (anupkg1985) commented Nov 10, 2024

Current LangChain-Postgres framework, the database table name is hardcoded and defaults to the public schema due to the absence of a specified schema name. This limitation affects the flexibility of the PGVector connector, restricting its ability to dynamically configure during runtime. This poses significant challenges in multi-tenant architectures, where it's essential for different clients or projects to operate within separate schemas to ensure data isolation and management efficiency. Addressing this issue would involve enabling schema specification, thereby enhancing the connector's versatility and making it more suitable for diverse deployment scenarios.

Proposed solution -

  • Introduced new argument/ parameter table_schema to `PGVector' class
  • This new parameter is optional and defaulted to public schema

Discussions related to issue -
https://github.com/langchain-ai/langchain/discussions/20877
https://github.com/langchain-ai/langchain/discussions/18883

Example usage -

Default publicschema -

connection = "postgresql+psycopg://langchain:langchain@localhost:6024/langchain" 
collection_name = "my_docs"
embeddings = CohereEmbeddings()

vectorstore = PGVector(
    embeddings=embeddings,
    collection_name=collection_name,
    connection=connection,
    use_jsonb=True,
)

Specify custom table-schema -

connection = "postgresql+psycopg://langchain:langchain@localhost:6024/langchain" 
collection_name = "my_docs"
embeddings = CohereEmbeddings()

vectorstore = PGVector(
    embeddings=embeddings,
    collection_name=collection_name,
    connection=connection,
    use_jsonb=True,
    table_schema="custom"
)

@markobogoevski
Copy link
Copy Markdown

Will this get merged? Im in need of setting up pg vector tables per environment which I am isolating by the schema

@bradfordben
Copy link
Copy Markdown

I would also like this feature

@Ladvien
Copy link
Copy Markdown

Not having the ability to select a schema in a way that's idiomatic to langchain is problematic. Can this be merged?

@bradfordben
Copy link
Copy Markdown

Do you need to link reviewers to get this reviewed and merged?

@anupkg1985
Copy link
Copy Markdown
Author

Do you need to link reviewers to get this reviewed and merged?

Yes, could you please provide the link?

@LifeDJIK
Copy link
Copy Markdown

Table schema can already be specified using schema_translate_map from sqlalchemy:

engine_args={
    "execution_options": {
        "schema_translate_map": {
            None: "target_schema_name",
        },
    },
}

I think more useful will be to allow table names to be changed (like in #106).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants