Update docs

[cwaddingham]

Allow Backend to Override Pinecone index_name and namespace via Request Data

Overview

This PR enhances the flexibility of the backend by allowing API clients to specify the Pinecone index_name and namespace directly in the request payload or query parameters. If these values are provided in the request, they will be used for all Pinecone operations; otherwise, the backend will fall back to the default values set in environment variables.

Key Changes

• Request-level Overrides:
  • All relevant endpoints and backend logic now check for index_name and namespace in the incoming request data (payload, query params, or kwargs).
  • If present, these values override the environment variable defaults for Pinecone operations.
• Refactored Utility Functions:
  • Updated Pinecone utility functions (e.g., get_index) to accept and use request-level overrides.
• Endpoints Updated:
  • All ingestion, collection, and document endpoints in both ingestor_server and rag_server now support dynamic index/namespace selection.
• Backward Compatibility:
  • If no override is provided, the backend continues to use the environment variable configuration as before.

Motivation

• Enables multi-tenant and dynamic use cases where different API calls may need to operate on different Pinecone indexes or namespaces.
• Improves developer experience and flexibility for advanced workflows and integrations.

Testing

• Verified that requests specifying index_name and/or namespace use the provided values.
• Confirmed that requests without overrides continue to use the environment defaults.
• Ensured all existing tests pass and added new tests for override scenarios.

[Copilot]

Pull Request Overview

This PR enhances backend flexibility by allowing API clients to override the default Pinecone index and namespace using request data. Key changes include updating utility functions to accept override values, modifying server startup and ingestion methods accordingly, and updating documentation and notebooks to reflect the switch from Milvus to Pinecone.

• Updated Pinecone utility functions (get_index, create_index, add_documents) to use request-level overrides.
• Modified server and ingestion endpoints in rag_server and ingestor_server to support dynamic index/namespace values.
• Updated notebooks and docs to reflect the Pinecone changes and improved API usage instructions.

Reviewed Changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
src/utils/pinecone_utils.py	Updated functions to accept **kwargs override values for index_name and namespace.
src/rag_server/server.py	Modified the startup_event to use request overrides for index_name.
src/ingestor_server/main.py	Refactored functions to adopt index/namespace parameters instead of collection names.
notebooks/launchable.ipynb	Updated git clone URL and installation instructions from NVIDIA to Pinecone.
notebooks/ingestion_api_usage.ipynb	Updated API usage examples and request payloads to align with Pinecone.
docs/quickstart.md	Revised documentation to include Pinecone API key instructions and updated wording.

Comments suppressed due to low confidence (1)

notebooks/ingestion_api_usage.ipynb:330

• The variable 'pc' is used but not defined within the scope of the delete_documents function. Obtain the Pinecone client instance prior to using it (e.g., by calling get_pinecone_client()).

    "    index = pc.Index(index)\n",

[Copilot]

The payload passed as json is the function 'create_index' instead of a proper data object. Ensure that a valid JSON serializable dictionary is provided to the API endpoint.

Suggested change

	" async with session.post(f\"{BASE_URL}/v1/collections\", params=params, json=create_index, headers=HEADERS) as response:\n",
	" async with session.post(f\"{BASE_URL}/v1/collections\", params=params, json=params, headers=HEADERS) as response:\n",

[Copilot]

The variable 'fetched_count' is used without prior initialization. Initialize and update 'fetched_count' appropriately in the pagination loop.

Suggested change

	" if fetched_count is None:\n",
	" # Pinecone does not return total count, so we estimate as we go\n",
	" fetched_count = len(vectors)\n",
	" fetched_count += len(vectors)\n",

[Copilot]

The function delete_namespaces expects a parameter 'namespaces' but is being called with 'namespace'. Update the call to use the correct parameter name for consistency with the function definition.

Suggested change

	"await delete_namespaces(index=\"nvidia-rag-index\", namespace=\"multimodal_data\")"
	"await delete_namespaces(index=\"nvidia-rag-index\", namespaces=[\"multimodal_data\"])"