diff --git a/multimodal-understanding/repeatable-patterns/33 - nova-grounding/nova_web_grounding_tutorial.ipynb b/multimodal-understanding/repeatable-patterns/33 - nova-grounding/nova_web_grounding_tutorial.ipynb new file mode 100644 index 0000000..aefeaf7 --- /dev/null +++ b/multimodal-understanding/repeatable-patterns/33 - nova-grounding/nova_web_grounding_tutorial.ipynb @@ -0,0 +1,523 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "50f371b9107e7d5f", + "metadata": {}, + "source": [ + "# Amazon Nova Web Grounding with Bedrock converse and converse_stream APIs\n", + "\n", + "Amazon Nova Web Grounding enhances Nova models by connecting them to real-time information beyond their knowledge cutoff, which results in more accurate and reliable responses. This feature is provided by the systemTool parameter when using the Bedrock Converse API and Invoke APIs. \n", + "\n", + "This tutorial walks through how to setup Nova Web Grounding and use it with the Amazon Bedrock converse and converse_stream APIs. \n", + "\n", + "## Step 1\n", + "- Log into your AWS account\n", + "- Go to Bedrock in your console\n", + "- Select Region - us-east-1\n", + "- Select Model Access from side tab\n", + "- Enable Nova Premier access" + ] + }, + { + "cell_type": "markdown", + "id": "a4757acf5a0c5a2c", + "metadata": {}, + "source": [ + "## Step 2: Configure AWS CLI with Default Profile\n", + "\n", + "### Do this step if you don't have credentials\n", + "#### Getting Your AWS Credentials\n", + "\n", + "1. Go to IAM ā†’ Users ā†’ Create User\n", + "2. Give a suitable name for the user and hit `Next`\n", + "3. Add relevant permissions to enable Bedrock access and finish setting up the user.\n", + " * bedrock:ListFoundationModels, bedrock:InvokeModel or use AmazonBedrockFullAccess if your security policies allow\n", + "4. Once the user is created, go to Security credentials tab\n", + "5. Click -> Create access key\n", + "6. Choose Command Line Interface (CLI) and Create key\n", + "7. Copy the Access Key ID and Secret Access Key\n", + "\n", + "### Configure AWS Profile\n", + "Open your choice of terminal and Run this command.\n", + "```bash\n", + "aws configure --profile nova-grounding-test-profile\n", + "```\n", + "\n", + "You'll be asked for:\n", + "- AWS Access Key ID\n", + "- AWS Secret Access Key\n", + "- Default region name (enter: us-east-1)\n", + "- Default Format type (optionally you can enter: json)\n", + "\n", + "If you decide to change the name of the profile then update the client below accordingly" + ] + }, + { + "cell_type": "markdown", + "id": "34bbea82788a24b7", + "metadata": {}, + "source": [ + "## Step 3\n", + "Install the python SDK. Run the below code only once and that will install the required packages automatically." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1f6638ab0e8f2971", + "metadata": {}, + "outputs": [], + "source": [ + "# We will first install Python SDK. You need to run this only one time\n", + "!pip install boto3" + ] + }, + { + "cell_type": "markdown", + "id": "2b21cac6b355afe", + "metadata": {}, + "source": [ + "## Step 4: Run the below code\n", + "Below code sample will create the function which can then be called with a query. You need to only run this once unless you erase the context in which case you will have to run this again." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "initial_id", + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "import json\n", + "import boto3\n", + "import time\n", + "from botocore.config import Config\n", + "\n", + "# Configure timeout settings for the Bedrock API client\n", + "# These settings help handle network issues and long-running requests\n", + "boto_config = Config(\n", + " read_timeout=300, # Maximum time to wait for a response (5 minutes)\n", + " connect_timeout=10, # Maximum time to wait for connection establishment (10 seconds)\n", + " retries={'max_attempts': 2} # Number of retry attempts if the request fails\n", + ")\n", + "\n", + "def ask_bedrock_converse(question, model_id=\"us.amazon.nova-premier-v1:0\", endpoint=\"https://bedrock-runtime.us-east-1.amazonaws.com\"):\n", + " \"\"\"\n", + " Send a question to Bedrock using the non-streaming Converse API and get the complete response.\n", + "\n", + " Args:\n", + " question (str): The question to ask the AI model\n", + " model_id (str): The specific Bedrock model to use (default: Nova Premier)\n", + " endpoint (str): The Bedrock API endpoint URL\n", + " \"\"\"\n", + " try:\n", + " # Create a Bedrock runtime client with our configuration\n", + " # This client will handle communication with AWS Bedrock service\n", + " session = boto3.Session(region_name='us-east-1',profile_name='nova-grounding-test-profile') # CHANGEME: Credential profile corresponding to allow-listed account and with bedrock access\n", + " client = session.client(\"bedrock-runtime\", region_name=\"us-east-1\", endpoint_url=endpoint, config=boto_config)\n", + "\n", + " # Prepare the conversation in the format expected by Bedrock\n", + " # This follows the conversational AI format with roles and content\n", + " conversation = [\n", + " {\n", + " \"role\": \"user\", # Indicates this message is from the user\n", + " \"content\": [{\"text\": question}], # The actual question text\n", + " }\n", + " ]\n", + "\n", + " # Configure tools that the AI model can use\n", + " # In this case, we're enabling Nova Web Grounding functionality\n", + " toolConfiguration = {\n", + " \"tools\": [\n", + " {\n", + " \"systemTool\": {\n", + " \"name\": \"nova_grounding\" # Enables the model to search for real-time information\n", + " }\n", + " }\n", + " ]\n", + " }\n", + "\n", + " # Print the question and formatting for better readability\n", + " print(f\"Question: {question}\")\n", + " print(\"=\" * 50) # Visual separator\n", + " print(\"Response:\")\n", + " print(\"-\" * 30) # Another visual separator\n", + "\n", + " # Make the API call to Bedrock\n", + " response = client.converse(\n", + " modelId=model_id, # Which AI model to use\n", + " messages=conversation, # The conversation history (just our question)\n", + " toolConfig=toolConfiguration,\n", + " )\n", + "\n", + " # Extract and display the request ID for debugging/tracking purposes\n", + " request_id = response['ResponseMetadata']['RequestId']\n", + " print(f\"Request ID: {request_id}\")\n", + " print()\n", + "\n", + " # Simply print the response as JSON with nice formatting\n", + " print(f\"Response:\\n{json.dumps(response['output']['message'], indent=4)}\")\n", + " print(\"-\" * 80)\n", + " \n", + " # Return the response for use in other cells\n", + " return response\n", + "\n", + " except Exception as e:\n", + " # Handle any errors that occur during the API call or processing\n", + " print(f\"Error processing query: {e}\")\n", + " return None" + ] + }, + { + "cell_type": "markdown", + "id": "2dba335d344997e9", + "metadata": {}, + "source": [ + "## Step 5: Set the ModelId \n", + "Change this to any supported model in the Nova family. Nova Premier is supported initially. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "bae6bf4036aa505d", + "metadata": {}, + "outputs": [], + "source": [ + "# Specify which AI model to use - you can change this to other available models. \n", + "\n", + "model_id = \"us.amazon.nova-premier-v1:0\"\n" + ] + }, + { + "cell_type": "markdown", + "id": "formatted_response_section", + "metadata": {}, + "source": [ + "## Step 6: Execute a query and format the response (Clean text with citations)\n", + "Below code will format the raw response to show clean text without thinking tags and with proper citations" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "format_response_cell", + "metadata": {}, + "outputs": [], + "source": [ + "import re\n", + "\n", + "def format_bedrock_response(response):\n", + " \"\"\"\n", + " Format the Bedrock response to show clean text with citations\n", + " \n", + " Args:\n", + " response: Raw Bedrock response dictionary\n", + " \n", + " Returns:\n", + " Formatted text with inline citations and sources list\n", + " \"\"\"\n", + " if not response or 'output' not in response:\n", + " return \"No response to format.\"\n", + " \n", + " message = response['output']['message']\n", + " content = message.get('content', [])\n", + " \n", + " result_parts = []\n", + " citation_counter = 1\n", + " all_citations = {}\n", + " \n", + " # Process content items sequentially\n", + " i = 0\n", + " while i < len(content):\n", + " item = content[i]\n", + " \n", + " # Handle text content\n", + " if 'text' in item:\n", + " text = item['text']\n", + " \n", + " # Skip tool use items\n", + " if 'toolUse' in item:\n", + " i += 1\n", + " continue\n", + " \n", + " # Remove thinking tags completely\n", + " text = re.sub(r'.*?', '', text, flags=re.DOTALL)\n", + " text = re.sub(r']*>', '', text)\n", + " \n", + " # Clean up the text\n", + " text = text.strip()\n", + " \n", + " # Only process non-empty text\n", + " if text:\n", + " # Check if the next item is a citation for this text\n", + " citation_added = False\n", + " if i + 1 < len(content) and 'citationsContent' in content[i + 1]:\n", + " citation_item = content[i + 1]\n", + " citation_data = citation_item['citationsContent']\n", + " \n", + " if 'citations' in citation_data:\n", + " citations_for_this_text = []\n", + " for citation in citation_data['citations']:\n", + " if 'location' in citation and 'web' in citation['location']:\n", + " web_info = citation['location']['web']\n", + " url = web_info.get('url', '')\n", + " domain = web_info.get('domain', '')\n", + " \n", + " if url and domain:\n", + " citations_for_this_text.append((domain, url))\n", + " \n", + " # Add citation numbers to this text\n", + " if citations_for_this_text:\n", + " for domain, url in citations_for_this_text:\n", + " text += f'[{citation_counter}]'\n", + " all_citations[citation_counter] = (domain, url)\n", + " citation_counter += 1\n", + " citation_added = True\n", + " \n", + " # Skip the citation item since we processed it\n", + " if citation_added:\n", + " i += 1\n", + " \n", + " result_parts.append(text)\n", + " \n", + " i += 1\n", + " \n", + " # Join all text parts with proper spacing\n", + " result = ''.join(result_parts)\n", + " \n", + " # Clean up any double spaces or formatting issues\n", + " result = re.sub(r'\\s+', ' ', result)\n", + " result = re.sub(r'\\n\\s*\\n', '\\n\\n', result)\n", + " \n", + " # Add sources section if we have citations\n", + " if all_citations:\n", + " result += \"\\n\\n**Sources:**\\n\"\n", + " for num, (domain, url) in all_citations.items():\n", + " result += f\"{num}. {domain}: {url}\\n\"\n", + " \n", + " # Final cleanup\n", + " result = re.sub(r'\\n\\n\\n+', '\\n\\n', result)\n", + " result = result.strip()\n", + " \n", + " return result\n", + "\n", + "print(\"āœ… Response formatting function defined successfully!\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "test_new_question", + "metadata": {}, + "outputs": [], + "source": [ + "# Test with a question and format the response\n", + "question = \"What are the latest developments in renewable energy technology?\"\n", + "print(f\"šŸ” Testing with new question: {question}\")\n", + "print(\"=\" * 60)\n", + "\n", + "# Get raw response\n", + "response = ask_bedrock_converse(question, model_id)\n", + "\n", + "# Format and display\n", + "if response:\n", + " print(\"\\nšŸ“ Formatted Response:\")\n", + " print(\"=\" * 50)\n", + " formatted = format_bedrock_response(response)\n", + " print(formatted)\n", + " print(\"=\" * 50)\n", + "else:\n", + " print(\"āŒ Failed to get response for new question.\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "db6a9ca7", + "metadata": {}, + "outputs": [], + "source": [ + "# Test with another question and format the response\n", + "new_question = \"What are latest advancements in AI ?\"\n", + "print(f\"šŸ” Testing with new question: {new_question}\")\n", + "print(\"=\" * 60)\n", + "\n", + "# Get raw response\n", + "new_response = ask_bedrock_converse(new_question, model_id)\n", + "\n", + "# Format and display\n", + "if new_response:\n", + " print(\"\\nšŸ“ Formatted Response:\")\n", + " print(\"=\" * 50)\n", + " formatted_new = format_bedrock_response(new_response)\n", + " print(formatted_new)\n", + " print(\"=\" * 50)\n", + "else:\n", + " print(\"āŒ Failed to get response for new question.\")" + ] + }, + { + "cell_type": "markdown", + "id": "63c74010", + "metadata": {}, + "source": [ + "## Step 7: Now let's stream the model response\n", + "\n", + "We will use the Bedrock converse stream API to stream the model response. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "4cd38436", + "metadata": {}, + "outputs": [], + "source": [ + "import boto3\n", + "import time\n", + "from botocore.config import Config\n", + "\n", + "# Configure timeout settings for the Bedrock API client\n", + "# These settings help handle network issues and long-running requests\n", + "boto_config = Config(\n", + " read_timeout=300, # Maximum time to wait for a response (5 minutes)\n", + " connect_timeout=10, # Maximum time to wait for connection establishment (10 seconds)\n", + " retries={'max_attempts': 2} # Number of retry attempts if the request fails\n", + ")\n", + "\n", + "def stream_bedrock_response(question, model_id=\"us.amazon.nova-premier-v1:0\", endpoint=\"https://bedrock-runtime.us-east-1.amazonaws.com\"):\n", + " \"\"\"\n", + " Stream response from Bedrock API and print chunks as they arrive\n", + "\n", + " Args:\n", + " question (str): The question to ask the AI model\n", + " model_id (str): The specific Bedrock model to use (default: Nova Premier)\n", + " endpoint (str): The Bedrock API endpoint URL\n", + " \"\"\"\n", + " try:\n", + " # Create a Bedrock runtime client with our configuration\n", + " # This client will handle communication with AWS Bedrock service\n", + " session = boto3.Session(region_name='us-east-1',profile_name='nova-grounding-test-profile') # CHANGEME: Credential profile corresponding to allow-listed account and with bedrock access\n", + " client = session.client(\"bedrock-runtime\", region_name=\"us-east-1\", endpoint_url=endpoint, config=boto_config)\n", + "\n", + " # Prepare the conversation in the format expected by Bedrock\n", + " # This follows the conversational AI format with roles and content\n", + " conversation = [\n", + " {\n", + " \"role\": \"user\", # Indicates this message is from the user\n", + " \"content\": [{\"text\": question}], # The actual question text\n", + " }\n", + " ]\n", + "\n", + " # Configure tools that the AI model can use\n", + " # In this case, we're enabling Nova Web Grounding functionality\n", + " toolConfiguration = {\n", + " \"tools\": [\n", + " {\n", + " \"systemTool\": {\n", + " \"name\": \"nova_grounding\" # Enables the model to search for real-time information\n", + " }\n", + " }\n", + " ]\n", + " }\n", + "\n", + " # Print the question and formatting for better readability\n", + " print(f\"Question: {question}\")\n", + " print(\"=\" * 50) # Visual separator\n", + " print(\"Streaming response:\")\n", + " print(\"-\" * 30) # Another visual separator\n", + "\n", + " # Make the streaming API call to Bedrock\n", + " # converse_stream returns chunks of the response as they're generated\n", + " streaming_response = client.converse_stream(\n", + " modelId=model_id, # Which AI model to use\n", + " messages=conversation, # The conversation history (just our question)\n", + " toolConfig=toolConfiguration # Tools the model can use\n", + " )\n", + "\n", + " # Extract and display the request ID for debugging/tracking purposes\n", + " request_id = streaming_response['ResponseMetadata']['RequestId']\n", + " print(f\"Request ID: {request_id}\")\n", + " print()\n", + "\n", + " # Process the streaming response chunk by chunk\n", + " # This allows us to display text as it's being generated, not just at the end\n", + " for chunk in streaming_response[\"stream\"]:\n", + " # Check if this chunk contains content (text, tool use, etc.)\n", + " if \"contentBlockDelta\" in chunk:\n", + " # Extract the actual content from the chunk\n", + " delta = chunk[\"contentBlockDelta\"][\"delta\"]\n", + "\n", + " # Handle different types of content\n", + " if \"text\" in delta:\n", + " # Regular text content - print immediately without newline\n", + " # end=\"\" prevents automatic newline, flush=True ensures immediate display\n", + " print(delta[\"text\"], end=\"\", flush=True)\n", + " elif \"toolUse\" in delta:\n", + " # Model is using a tool (like nova_grounding) - show this activity\n", + " print(f\"\\n[TOOL_USE: {delta['toolUse']}]\\n\", end=\"\", flush=True)\n", + " elif \"citation\" in delta:\n", + " # Model is providing citations for its sources - display them\n", + " print(f\"\\n[CITATION: {delta['citation']}]\\n\", end=\"\", flush=True)\n", + "\n", + " # Print completion message and formatting\n", + " print(\"\\n\" + \"=\" * 50)\n", + " print(\"Streaming complete!\")\n", + "\n", + " except Exception as e:\n", + " # Handle any errors that occur during the API call or processing\n", + " print(f\"Error processing query: {e}\")" + ] + }, + { + "cell_type": "markdown", + "id": "4834427a", + "metadata": {}, + "source": [ + "## Step 8: Try streaming with a query" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f3fd0be7", + "metadata": {}, + "outputs": [], + "source": [ + "question = \"What is today's market trends?\"\n", + "\n", + "# Specify which AI model to use - you can change this to other available models\n", + "# Nova Premier is Amazon's latest and most capable model\n", + "model_id = \"us.amazon.nova-premier-v1:0\"\n", + "\n", + "# Call the streaming function with our question and model\n", + "stream_bedrock_response(question, model_id)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Nova-Grounding-Set-up Pack", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.13.4" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +}