|
27 | 27 | "jp-MarkdownHeadingCollapsed": true |
28 | 28 | }, |
29 | 29 | "source": [ |
30 | | - "\n", |
31 | | - "# 1. Create and Deploy Your Free Tier Operational cluster on Capella,\n", |
| 30 | + "# 1. Create and Deploy Your Free Tier Operational Cluster on Capella\n", |
32 | 31 | " To get started with Couchbase Capella, create an account and use it to deploy a cluster. To know more, please follow the [instructions](https://docs.couchbase.com/cloud/get-started/create-account.html).\n", |
33 | | - " ### Couchbase Capella Configuration,\n", |
34 | | - " When running Couchbase using [Capella](https://cloud.couchbase.com/sign-in), the following prerequisites need to be met.\n", |
| 32 | + " ### Couchbase Capella Configuration\n", |
| 33 | + " When running Couchbase using [Capella](https://cloud.couchbase.com/sign-in), the following prerequisites need to be met:\n", |
35 | 34 | " * Create the [database credentials](https://docs.couchbase.com/cloud/clusters/manage-database-users.html) to access the travel-sample bucket (Read and Write) used in the application.\n", |
36 | 35 | " * [Allow access](https://docs.couchbase.com/cloud/clusters/allow-ip-address.html) to the Cluster from the IP on which the application is running." |
37 | 36 | ] |
|
41 | 40 | "id": "4369c925-adbc-4c7d-9ea6-04ff020cb1a6", |
42 | 41 | "metadata": {}, |
43 | 42 | "source": [ |
44 | | - "\n", |
45 | 43 | "# 2. Data Upload and Preparation\n", |
46 | 44 | "\n", |
47 | | - "There are various techniques which exists to insert the data in the cluster, to read about the techniques please follow the [sample-data import](https://docs.couchbase.com/cloud/clusters/data-service/import-data-documents.html#import-sample-data) guide.\n", |
| 45 | + "There are various techniques that exist to insert data into the cluster. To read about the techniques, please follow the [sample-data import](https://docs.couchbase.com/cloud/clusters/data-service/import-data-documents.html#import-sample-data) guide.\n", |
48 | 46 | "\n", |
49 | | - "After data upload is comlete, follow the next steps to achieve vectorization for your required fields.\n" |
| 47 | + "After data upload is complete, follow the next steps to achieve vectorization for your required fields." |
50 | 48 | ] |
51 | 49 | }, |
52 | 50 | { |
|
55 | 53 | "metadata": {}, |
56 | 54 | "source": [ |
57 | 55 | "# 3. Deploying the Model\n", |
58 | | - "Now, before we actually create embedding for the documents we need to deploy a model which will create the embedding for us.\n", |
59 | | - "## 3.1: Selecting the model \n", |
60 | | - "1. To select the model, you first need to navigate to the \"<B>AI Services</B>\" tab, then selecting \"<B>Models</B>\" and clicking on \"<B>Deploy New Model</B>\"\n", |
| 56 | + "Now, before we actually create embeddings for the documents, we need to deploy a model that will create the embeddings for us.\n", |
| 57 | + "## 3.1: Selecting the Model \n", |
| 58 | + "1. To select the model, you first need to navigate to the \"<B>AI Services</B>\" tab, then select \"<B>Models</B>\" and click on \"<B>Deploy New Model</B>\".\n", |
61 | 59 | " \n", |
62 | 60 | " <img src=\"./img/importing_model.png\" width=\"950px\" height=\"500px\" style=\"padding: 5px; border-radius: 10px 20px 30px 40px; border: 2px solid #555;\">\n", |
63 | 61 | "\n", |
64 | | - "3. Enter the <B>model name</B>, and choose the model that you want to deploy. After Selecting your model, choose the <B>model infrastructure</B> and <B>region</B> where the model will be deployed.\n", |
| 62 | + "2. Enter the <B>model name</B>, and choose the model that you want to deploy. After selecting your model, choose the <B>model infrastructure</B> and <B>region</B> where the model will be deployed.\n", |
65 | 63 | " \n", |
66 | 64 | " <img src=\"./img/deploying_model.png\" width=\"800px\" height=\"800px\" style=\"padding: 5px; border-radius: 10px 20px 30px 40px; border: 2px solid #555;\">\n", |
67 | 65 | "\n", |
68 | | - "## 3.2 Access control to the model\n", |
| 66 | + "## 3.2 Access Control to the Model\n", |
69 | 67 | "\n", |
70 | | - "1. After deploying the model, go to the \"<B>Models</B>\" tab in the <B>AI-services</B> and click on \"<B>setup access</B>\".\n", |
| 68 | + "1. After deploying the model, go to the \"<B>Models</B>\" tab in the <B>AI Services</B> and click on \"<B>Setup Access</B>\".\n", |
71 | 69 | "\n", |
72 | 70 | " <img src=\"./img/model_setup_access.png\" width=\"1100px\" height=\"400px\" style=\"padding: 5px; border-radius: 10px 20px 30px 40px; border: 2px solid #555;\">\n", |
73 | 71 | "\n", |
74 | | - "3. Enter your <B>api_key_name</B>, <B>expiration time</B> and the <B>IP-address</B> from which you will be accessing the model.\n", |
| 72 | + "2. Enter your <B>API key name</B>, <B>expiration time</B> and the <B>IP address</B> from which you will be accessing the model.\n", |
75 | 73 | "\n", |
76 | 74 | " <img src=\"./img/model_api_key_form.png\" width=\"1100px\" height=\"600px\" style=\"padding: 5px; border-radius: 10px 20px 30px 40px; border: 2px solid #555;\">\n", |
77 | 75 | "\n", |
78 | | - "4. Download your API key\n", |
| 76 | + "3. Download your API key\n", |
79 | 77 | "\n", |
80 | 78 | " <img src=\"./img/download_api_key_details.png\" width=\"1200px\" height=\"800px\" style=\"padding: 5px; border-radius: 10px 20px 30px 40px; border: 2px solid #555;\">" |
81 | 79 | ] |
|
87 | 85 | "source": [ |
88 | 86 | "# 4. Deploying AutoVectorization Workflow\n", |
89 | 87 | "\n", |
90 | | - "Now, we are in the step which will help us create the embeddings/vectors. To proceed with the process of craetion of vectorization please follow the steps below:\n", |
| 88 | + "Now, we are at the step that will help us create the embeddings/vectors. To proceed with the vectorization process, please follow the steps below:\n", |
91 | 89 | "\n", |
92 | | - "1. For deploying the autovectorization, you need to go to the <B>`ai-services`</B> tab, then click on the <B>`workflows`</B>, and then click on <B>`Create New Workflow`</B>.\n", |
| 90 | + "1. For deploying the autovectorization, you need to go to the <B>`AI Services`</B> tab, then click on <B>`Workflows`</B>, and then click on <B>`Create New Workflow`</B>.\n", |
93 | 91 | "\n", |
94 | 92 | " <img src=\"./img/workflow.png\" width=\"1000px\" height=\"500px\" style=\"padding: 5px; border-radius: 10px 20px 30px 40px; border: 2px solid #555;\">\n", |
95 | 93 | " \n", |
96 | | - "2. Start your workflow deployment by giving it a name, and selecting from where your data will be provided to the auto-vectorization service. There are currently 3 options, <B>`pre-processed data(JSON format) from capella`</B>, <B>`pre-processed data(JSON format) from external sources(S3 buckets)`</B> and <B>`unstructured data from external sources (S3 buckets)`</B>. For this tutorial we will be choosing first option which is pre-processed data from capella.\n", |
| 94 | + "2. Start your workflow deployment by giving it a name and selecting where your data will be provided to the auto-vectorization service. There are currently 3 options: <B>`pre-processed data (JSON format) from Capella`</B>, <B>`pre-processed data (JSON format) from external sources (S3 buckets)`</B> and <B>`unstructured data from external sources (S3 buckets)`</B>. For this tutorial, we will choose the first option, which is pre-processed data from Capella.\n", |
97 | 95 | "\n", |
98 | 96 | " <img src=\"./img/start_workflow.png\" width=\"1000px\" height=\"500px\" style=\"padding: 5px; border-radius: 10px 20px 30px 40px; border: 2px solid #555;\">\n", |
99 | 97 | "\n", |
100 | 98 | "3. Now, select the <B>`cluster`</B>, <B>`bucket`</B>, <B>`scope`</B> and <B>`collection`</B> from which you want to select the documents and get the data vectorized.\n", |
101 | 99 | "\n", |
102 | 100 | " <img src=\"./img/vector_data_source.png\" width=\"1000px\" height=\"500px\" style=\"padding: 5px; border-radius: 10px 20px 30px 40px; border: 2px solid #555;\">\n", |
103 | 101 | "\n", |
104 | | - "4. <B>Field Mapping</B> will be used to tell the AutoVectorize service that which data will be converted to embeddings.\n", |
| 102 | + "4. <B>Field Mapping</B> will be used to tell the AutoVectorize service which data will be converted to embeddings.\n", |
105 | 103 | "\n", |
106 | | - " There are two options:-\n", |
| 104 | + " There are two options:\n", |
107 | 105 | "\n", |
108 | 106 | " - <B>All source fields</B> - This feature will convert all your fields inside the document to a single vector field.\n", |
109 | 107 | " \n", |
110 | 108 | " <img src=\"./img/vector_all_field_mapping.png\" width=\"900px\" height=\"400px\" style=\"padding: 5px; border-radius: 10px 20px 30px 40px; border: 2px solid #555;\">\n", |
111 | 109 | "\n", |
112 | 110 | "\n", |
113 | | - " - <B>Custom source fields</B> - This feature will convert specific fields which are chosen by the user to a single vector field, in the image below we have chosen <B>`address`</B>, <B>`description`</B> and <B>`id`</B> as the fields to be converted to a vector having the name as <B>`vec_addr_decr_id_mapping`</B>.\n", |
| 111 | + " - <B>Custom source fields</B> - This feature will convert specific fields chosen by the user to a single vector field. In the image below, we have chosen <B>`address`</B>, <B>`description`</B> and <B>`id`</B> as the fields to be converted to a vector with the name <B>`vec_addr_decr_id_mapping`</B>.\n", |
114 | 112 | " \n", |
115 | 113 | " <img src=\"./img/vector_custom_field_mapping.png\" width=\"900px\" height=\"400px\" style=\"padding: 5px; border-radius: 10px 20px 30px 40px; border: 2px solid #555;\">\n", |
116 | 114 | " \n", |
117 | | - "5. After choosing your type of mapping, you will be required to either have an index on the new vector_embedding field or you can skip the creation of vector index which is not recommended as you will be losing out the functionality of vector searching.\n", |
| 115 | + "5. After choosing your type of mapping, you will be required to either create an index on the new vector_embedding field or you can skip the creation of a vector index, which is not recommended as you will lose the functionality of vector searching.\n", |
118 | 116 | "\n", |
119 | 117 | " <img src=\"./img/vector_index.png\" width=\"1200px\" height=\"500px\" style=\"padding: 5px; border-radius: 10px 20px 30px 40px; border: 2px solid #555;\">\n", |
120 | 118 | "\n", |
121 | | - "6. All the steps mentioned above.\n", |
| 119 | + "6. Review all the steps mentioned above, below screenshot highlights the whole process of deploying AutoVectorization workflow.\n", |
122 | 120 | "\n", |
123 | 121 | " <img src=\"./img/vector_index_page.png\" width=\"1200px\" height=\"1200px\" style=\"padding: 5px; border-radius: 10px 20px 30px 40px; border: 2px solid #555;\">\n", |
124 | 122 | " \n", |
125 | | - "- After this step your vector-embedding for the selected fields must be ready and you can check it out in the capella UI. Now, in the next step we will demonsterate how we can use the generated vectors to do the vector search." |
| 123 | + "After this step, your vector embeddings for the selected fields should be ready, and you can check them out in the Capella UI. In the next step, we will demonstrate how we can use the generated vectors to perform vector search." |
126 | 124 | ] |
127 | 125 | }, |
128 | 126 | { |
|
169 | 167 | "metadata": {}, |
170 | 168 | "outputs": [], |
171 | 169 | "source": [ |
172 | | - "endpoint = \"couchbases://cb.xyz.com\" # Replace this with Connection String\n", |
| 170 | + "endpoint = \"couchbases://cb.xyz.com\" # Replace this with Connection String\n", |
173 | 171 | "username = \"YOUR_USERNAME\" # Replace this with your username\n", |
174 | 172 | "password = \"YOUR_PASSWORD\" # Replace this with your password\n", |
175 | 173 | "auth = PasswordAuthenticator(username, password)\n", |
176 | | - "# Configure cluster options with SSL verification disabled for testing, in production you should enable it\n", |
| 174 | + "# Configure cluster options with SSL verification disabled for testing; in production you should enable it\n", |
177 | 175 | "options = ClusterOptions(auth, tls_verify='none')\n", |
178 | 176 | "options.apply_profile(\"wan_development\")\n", |
179 | 177 | "cluster = Cluster(endpoint, options)" |
|
200 | 198 | "bucket_name = \"travel-sample\"\n", |
201 | 199 | "scope_name = \"inventory\"\n", |
202 | 200 | "collection_name = \"hotel\"\n", |
203 | | - "index_name = \"hybrid_autovec_workflow_vec_addr_descr_id\" # This is the name of the search index which was created in the step 4.5 and can also be seen in the search tab of the cluster.\n", |
204 | | - " # It shall be noted that hybrid_workflow_name_index_fieldname is the naming convention for the index created by AutoVectorization workflow where\n", |
| 201 | + "index_name = \"hybrid_autovec_workflow_vec_addr_descr_id\" # This is the name of the search index that was created in step 4.5 and can also be seen in the search tab of the cluster.\n", |
| 202 | + " # It should be noted that hybrid_workflow_name_index_fieldname is the naming convention for the index created by AutoVectorization workflow where\n", |
205 | 203 | " # fieldname is the name of the field being indexed.\n", |
206 | 204 | "embedder = NVIDIAEmbeddings(\n", |
207 | | - " model=\"nvidia/nv-embedqa-e5-v5\", # This is the model which will be used to create the embedding of the query.\n", |
208 | | - " api_key=\"nvapi-xyz\" # This is the api key using which your model will be accessed.\n", |
| 205 | + " model=\"nvidia/nv-embedqa-e5-v5\", # This is the model that will be used to create the embedding of the query.\n", |
| 206 | + " api_key=\"nvapi-xyz\" # This is the API key that will be used to access your model.\n", |
209 | 207 | ")" |
210 | 208 | ] |
211 | 209 | }, |
|
237 | 235 | " collection_name=collection_name,\n", |
238 | 236 | " embedding=embedder,\n", |
239 | 237 | " index_name=index_name,\n", |
240 | | - " text_key=\"address\", # your document's text field\n", |
241 | | - " embedding_key=\"vec_addr_descr_id\" # this is the field in which your vector(embedding) is stored in the cluster.\n", |
| 238 | + " text_key=\"address\", # Your document's text field\n", |
| 239 | + " embedding_key=\"vec_addr_descr_id\" # This is the field in which your vector (embedding) is stored in the cluster.\n", |
242 | 240 | ")" |
243 | 241 | ] |
244 | 242 | }, |
|
257 | 255 | }, |
258 | 256 | { |
259 | 257 | "cell_type": "code", |
260 | | - "execution_count": 18, |
| 258 | + "execution_count": null, |
261 | 259 | "id": "177fd6d5", |
262 | 260 | "metadata": {}, |
263 | 261 | "outputs": [ |
|
275 | 273 | "query = \"Woodhead Road\"\n", |
276 | 274 | "results = vector_store.similarity_search(query, k=3)\n", |
277 | 275 | "\n", |
278 | | - "# Printing out the top-k results\n", |
| 276 | + "# Print out the top-k results\n", |
279 | 277 | "for rank, doc in enumerate(results, start=1):\n", |
280 | 278 | " title = doc.metadata.get(\"title\", \"<no title>\")\n", |
281 | 279 | " address_text = doc.page_content\n", |
|
289 | 287 | "source": [ |
290 | 288 | "## 6. Results and Interpretation\n", |
291 | 289 | "\n", |
292 | | - "As we can see 3 (or `k`) ranked results printed in the output.\n", |
| 290 | + "As we can see, 3 (or `k`) ranked results are printed in the output.\n", |
293 | 291 | "\n", |
294 | 292 | "### What Each Part Means\n", |
295 | | - "- Leading number (1,2,3): The result rank (1 = most similar to your query).\n", |
| 293 | + "- Leading number (1, 2, 3): The result rank (1 = most similar to your query).\n", |
296 | 294 | "- Title: Pulled from `doc.metadata.get(\"title\", \"<no title>\")`. If your documents don't contain a `title` field, you will see `<no title>`.\n", |
297 | 295 | "- Address text: This is the value of the field you configured as `text_key` (in this tutorial: `address`). It represents the human-readable content we chose to display.\n", |
298 | 296 | "\n", |
299 | | - "### How The Ranking Works\n", |
| 297 | + "### How the Ranking Works\n", |
300 | 298 | "1. Your natural language query (e.g., `\"Woodhead Road\"`) is embedded using the NVIDIA model (`nvidia/nv-embedqa-e5-v5`).\n", |
301 | 299 | "2. The vector store compares the query embedding to stored document embeddings in the field you configured (`embedding_key = \"vec_addr_descr_id\"`).\n", |
302 | | - "3. Results are sorted by vector similarity. Higher similarity = nearer semantic meaning.\n", |
| 300 | + "3. Results are sorted by vector similarity. Higher similarity = closer semantic meaning.\n", |
303 | 301 | "\n", |
304 | 302 | "\n", |
305 | | - "> Your vector search pipeline is working if the returned documents feel meaningfully related to your natural language query—even when exact keywords do not match. Feel free to experiment with increasingly descriptive queries to observe the semantic power of the embeddings.\n" |
| 303 | + "> Your vector search pipeline is working if the returned documents feel meaningfully related to your natural language query—even when exact keywords do not match. Feel free to experiment with increasingly descriptive queries to observe the semantic power of the embeddings." |
306 | 304 | ] |
307 | 305 | } |
308 | 306 | ], |
|
0 commit comments