Autovectorization Tutorial

[giriraj-singh-couchbase]

This guide is a comprehensive tutorial demonstrating how to use Couchbase Capella's AI Services auto-vectorization feature to automatically convert your data into vector embeddings and perform semantic search using LangChain.

📋 Overview

The main tutorial is contained in the Jupyter notebook autovec_langchain.ipynb, which walks you through:

1. Couchbase Capella Setup - Creating account, cluster, and access controls
2. Data Upload & Processing - Using sample data
3. Model Deployment - Deploying embedding models for vectorization
4. Auto-Vectorization Workflow - Setting up automated embedding generation
5. LangChain Integration - Building semantic search applications with vector similarity

[gemini-code-assist]

Summary of Changes

Hello @giriraj-singh-couchbase, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request delivers a new, detailed tutorial designed to guide users through the process of leveraging Couchbase Capella's AI Services for automatic data vectorization. The tutorial provides a complete walkthrough, from initial Capella account and cluster setup to deploying embedding models and configuring auto-vectorization workflows, culminating in practical examples of semantic search using LangChain. The aim is to empower users to easily transform their data into vector embeddings and build intelligent search applications.

Highlights

• New Auto-Vectorization Tutorial: Introduces a comprehensive tutorial demonstrating the use of Couchbase Capella's AI Services auto-vectorization feature to convert data into vector embeddings.
• LangChain Integration: The tutorial showcases how to perform semantic search using the generated vector embeddings by integrating with LangChain.
• Step-by-Step Guide: The tutorial covers essential steps including Couchbase Capella setup, data upload and processing, embedding model deployment, auto-vectorization workflow configuration, and practical LangChain integration examples.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[github-actions]

Caution

Notebooks or Frontmatter Files Have Been Modified

• Please ensure that a frontmatter.md file is accompanying the notebook file, and that the frontmatter is up to date.
• These changes will be published to the developer portal tutorials only if frontmatter.md is included.
• Proofread all changes before merging, as changes to notebook and frontmatter content will update the developer tutorial.

1 Notebook Files Modified:

Notebook File	Frontmatter Included?
autovec-structured/autovec_langchain.ipynb	✅

1 Frontmatter Files Modified:

Frontmatter File
autovec-structured/frontmatter.md
Note: frontmatter will be checked and tested in the Test Frontmatter workflow.

[gemini-code-assist]

Code Review

This pull request introduces a comprehensive tutorial on using Couchbase Capella's AI Services for auto-vectorization with LangChain. The tutorial is well-structured, but there are several areas for improvement to enhance clarity, correctness, and security for the end-user. My review includes feedback on the README file and the Jupyter notebook, addressing issues such as placeholder values, dependency management, broken links, inconsistent formatting, typos, and a hardcoded credential. Addressing these points will make the tutorial more polished and easier for users to follow.

[nithishr]

Can you apply the same comments as in #57 to this one as well?

[nithishr]

Same comments as in #57 are relevant here as well.

[nithishr]

Looks okay. A few things to make it more user friendly.

• Can you use the hierarchy of markdown (##, ###, ####) instead of numbers to organize the tutorial?
• The model has to be deployed in the same region as the database for workflows to work.
• Can you link to the model & workflows documentation in the relevant places?
• Imo, the combination of address, description & id is not a great field to vectorize. Address & id has no use in vector search. Only the description is meaningful.
• Can you show the document structure before & after the workfow is run?
• Will the tutorial work? The embedding field is different from the default

[giriraj-singh-couchbase]

Yes, the tutorial works.

[giriraj-singh-couchbase]

I will update with these comments. Thank you.

[giriraj-singh-couchbase]

Is there specific documentation for for model services? I was only able to get the AI-services tab where there is very little about the workflows.

[giriraj-singh-couchbase]

for the document structure I have added a line. Since, there is one field added at the end of the document I dont think the full image is needed.

[giriraj-singh-couchbase]

For the sake of ease I have used all source field into vector embeddings and updated accordingly.

[nithishr]

• Linking to docs (https://docs.couchbase.com/ai/build/vectorization-service/data-processing.html) in the relevant sections of Workflows would be nice.
• Please add this The model has to be deployed in the same region as the database for workflows to work. It is important as users could end up having to kill & redeploy clusters due to the region issue. It is also in the docs.

• For the sake of ease I have used all source field into vector embeddings and updated accordingly.
  Our aim should be to showcase simple realistic use cases not go for simplicity. In this dataset, it makes no sense to vectorize all fields into a single vector.

• Since, there is one field added at the end of the document I dont think the full image is needed.
  It is the only thing we are getting we are getting out of the Vectorization flow. So in my opinion, we should add it. It can be just a markdown of a sample document.

• There are some warnings due to Pydantic not being supported on Python 3.14. Can we get rid of this either by updating the langchain packages to 1.0.x or downgrading Python to 3.13 for the tutorial?
• The package versions for langchain-couchbase are not right. Version 0.5.0 & higher are the only ones that support QueryVectorStore.
• In the field mapping section, it is not mentioned which approach has been selected. It has to be inferred from the screenshots. You can include either just one approach screenshot or mention explicitly about the selected approach.

• After choosing the type of mapping, it is required to either create an index on the new vector_embedding field or the creation of a vector index can be skipped, which is not recommended as the functionality of vector searching will be lost.
  This statement is not correct. Vector search will work using brute force without an index.

• The comments in the Cluster setup code is a bit too off to the right. Can you reduce the spacing for the comments so that they are visible without scrolling?
• The page content in the results does not make much sense. You should include other relevant fields in the vector search response like reviews, name, etc. This is supported by the integration.

[nithishr]

There is also a test failure

TEST FAILURE
Entry: 
   Path: tutorial/markdown/generated/vector-search-cookbook/autovec-structured-autovec_langchain.md
   Title: Auto-Vectorization of Strucutured Data with Couchbase Capella AI Services
Invalid technology: 
    Artificial Intelligence 
Note: 
   Valid technologies: 
   [connectors,kv,query,capella,server,index,mobile,fts,sync gateway,eventing,analytics,udf,vector search,react,edge-server,app-services,hyperscale vector index,composite vector index]

[nithishr]

• Linking to docs (https://docs.couchbase.com/ai/build/vectorization-service/data-processing.html) in the relevant sections of Workflows would be nice.
• Please add this The model has to be deployed in the same region as the database for workflows to work. It is important as users could end up having to kill & redeploy clusters due to the region issue. It is also in the docs.

• For the sake of ease I have used all source field into vector embeddings and updated accordingly.
  Our aim should be to showcase simple realistic use cases not go for simplicity. In this dataset, it makes no sense to vectorize all fields into a single vector.

• Since, there is one field added at the end of the document I dont think the full image is needed.
  It is the only thing we are getting we are getting out of the Vectorization flow. So in my opinion, we should add it. It can be just a markdown of a sample document.

• There are some warnings due to Pydantic not being supported on Python 3.14. Can we get rid of this either by updating the langchain packages to 1.0.x or downgrading Python to 3.13 for the tutorial?
• The package versions for langchain-couchbase are not right. Version 0.5.0 & higher are the only ones that support QueryVectorStore.
• In the field mapping section, it is not mentioned which approach has been selected. It has to be inferred from the screenshots. You can include either just one approach screenshot or mention explicitly about the selected approach.

• After choosing the type of mapping, it is required to either create an index on the new vector_embedding field or the creation of a vector index can be skipped, which is not recommended as the functionality of vector searching will be lost.
  This statement is not correct. Vector search will work using brute force without an index.

• The comments in the Cluster setup code is a bit too off to the right. Can you reduce the spacing for the comments so that they are visible without scrolling?
• The page content in the results does not make much sense. You should include other relevant fields in the vector search response like reviews, name, etc. This is supported by the integration.

[deniswsrosa]

Minor change at the beggining

[deniswsrosa]

Link to a docs page (if there is any) that talk about these items.

[giriraj-singh-couchbase]

The link is there at the very starting.