fix(genai): extract and include metadata

[SYED-M-HUSSAIN]

PR Description

This pull request fixes an issue where grounding metadata from the Google search tool responses was not preserved. It ensures that fields such as grounding_supports, grounding_chunks, and web_search_queries are extracted and included in response_metadata, enabling citation tracking and improved traceability in search-augmented responses.

---

Relevant issues

Fixes #907

---

Type

🐛 Bug Fix

---

Changes (optional)

• Added _extract_grounding_metadata() helper function to safely extract grounding data.
• Updated _response_to_result() to include grounding_supports, grounding_chunks, and web_search_queries in response_metadata.
• Preserves backward compatibility with the existing response structure.

---

Testing (optional)

• Manually verified that metadata is preserved across multiple Google search tool responses.
• Confirmed presence of citation information in augmented responses.

---

Note (optional)

This fix improves the reliability of citations and source tracking in responses using Google search, supporting downstream systems relying on structured grounding metadata.

[SYED-M-HUSSAIN]

✅ Example Code

from google.ai.generativelanguage_v1beta.types import Tool as GenAITool
from langchain_google_genai import ChatGoogleGenerativeAI

# Define the LLM with Gemini 2.5 Flash preview model
llm = ChatGoogleGenerativeAI(model="gemini-2.5-flash-preview-04-17")

# Invoke the LLM with Google Search Tool enabled
resp = llm.invoke(
    "When is the next total solar eclipse in US?",
    tools=[GenAITool(google_search={})],
)

print(resp)

---

🔎 Before My Change

The model returned a response without grounded sources :

content='Okay, let\'s clarify based on the recent eclipse:\n\n1.  **The most recent total solar eclipse visible across a large portion of the contiguous US was on April 8, 2024.**\n\n2.  **The *absolute next* total solar eclipse visible in *any* US territory will be on March 30, 2033.** This one will be visible primarily in **Alaska**.\n\n3.  **The *next major* total solar eclipse to cross a significant portion of the *contiguous* United States (lower 48 states) will be on August 12, 2045.** This eclipse will have a path stretching from California to Florida.\n\nSo, depending on what you mean by "in US" (any territory vs. contiguous states), the answer is either **March 30, 2033 (Alaska)** or **August 12, 2045 (Contiguous US)**.' additional_kwargs={} response_metadata={'prompt_feedback': {'block_reason': 0, 'safety_ratings': []}, 'finish_reason': 'STOP', 'safety_ratings': []} id='run--d434c211-31c7-4dd5-9687-d61a55cadae9-0' usage_metadata={'input_tokens': 11, 'output_tokens': 203, 'total_tokens': 1102, 'input_token_details': {'cache_read': 0}}

---

🚀 After My Change

After enabling the google_search tool, the model response was grounded with real-time verified web sources and reflected improved factual accuracy:

content='The next total solar eclipse visible in the United States will occur on March 30, 2033, which will be visible over Alaska.\n\nFollowing the 2033 event, the contiguous United States will experience a total solar eclipse on August 23, 2044, with the path of totality crossing over Montana, North Dakota, and South Dakota. Another total solar eclipse with a coast-to-coast path across the US is anticipated on August 12, 2045, stretching from California to Florida.' additional_kwargs={} response_metadata={'prompt_feedback': {'block_reason': 0, 'safety_ratings': []}, 'finish_reason': 'STOP', 'model_name': 'models/gemini-2.5-flash-preview-04-17', 'safety_ratings': [], 'grounding_metadata': {'grounding_chunks': [{'web': {'uri': 'https://vertexaisearch.cloud.google.com/grounding-api-redirect/AbF9wXE1kr7ISat-otQwxzwCWm30cUU8DDPmFm8OF6oCTGaG0Y9CdRfH4L-ytp9B1cuj_9MFmpdd5yMJZsabEQTe8-DXeEL44aohfemCPElLpGdughHxse0FgkhNzkGIIg4hb_t4LWnLuTCXXmwcbuf1-wXHXMI=', 'title': 'wikipedia.org'}}, {'web': {'uri': 'https://vertexaisearch.cloud.google.com/grounding-api-redirect/AbF9wXGc7QDYHzpfGW7ZM1-fa3I297vfQZ1UBm6aQ5PmgJktm0ocY60MP21bjUNiB9sFTX_iZKCWIu8oy3iD8IkWOK7U-uFVMbIMALp1cpBS_TlS-Is6', 'title': 'nationaleclipse.com'}}, {'web': {'uri': 'https://vertexaisearch.cloud.google.com/grounding-api-redirect/AbF9wXHthFOEknQ_ubJRC31rMux4QVDyWtV2T1I9_UMnoLqb-ncjZ_NI3FxLMvZT8oXL9nTl4UnoR_AJAevje4DAJax7oilRv8_Aox2plp3HkD30Pg0KzOzRUMZHImMBsIrfR8QpEHsM8Q8xrsjD9jxCIp9j9kRK4sc7', 'title': 'cbsnews.com'}}], 'grounding_supports': [{'segment': {'start_index': 0, 'end_index': 122, 'text': 'The next total solar eclipse visible in the United States will occur on March 30, 2033, which will be visible over Alaska.'}, 'grounding_chunk_indices': [0, 1], 'confidence_scores': [1.0, 1.0]}, {'segment': {'start_index': 124, 'end_index': 317, 'text': 'Following the 2033 event, the contiguous United States will experience a total solar eclipse on August 23, 2044, with the path of totality crossing over Montana, North Dakota, and South Dakota.'}, 'grounding_chunk_indices': [2, 0, 1], 'confidence_scores': [1.0, 1.0, 1.0]}, {'segment': {'start_index': 318, 'end_index': 460, 'text': 'Another total solar eclipse with a coast-to-coast path across the US is anticipated on August 12, 2045, stretching from California to Florida.'}, 'grounding_chunk_indices': [2, 1], 'confidence_scores': [1.0, 1.0]}], 'web_search_queries': ['next total solar eclipse in US']}} id='run--8628b5bf-a2e7-4fd2-8a62-acf84c38c058-0' usage_metadata={'input_tokens': 11, 'output_tokens': 131, 'total_tokens': 343, 'input_token_details': {'cache_read': 0}, 'output_token_details': {'reasoning': 151}}

[mdrxy]

@SYED-M-HUSSAIN please resolve lint and conflicts

[mdrxy]

@SYED-M-HUSSAIN do you have any insight into why the original proto.Message.to_dict(candidate.grounding_metadata) isn't working?

[mdrxy]

addressed some regressions. note the floating point rounding being done - not sure why this was failing; I suspect issues on Google's end