vertax: enhanced support for Gemini structured output with OpenAPI schema objects

[minki-j]

PR Description

Problem

The current implementation of response_schema parameter in ChatVertexAI only supports Python dictionary schemas, which are automatically converted to gRPC Schema objects. This limitation prevents users from accessing advanced Gemini structured output features like Property Ordering, min/max items constraints, and other OpenAPI Schema specifications that require direct Schema object instantiation.

Solution

This PR enhances the response_schema parameter to accept both:

1. Dict schemas (existing functionality) - automatically converted to gRPC Schema objects
2. OpenAPI Schema objects (new functionality) - passed directly to the API without conversion

The implementation adds intelligent type checking in the _prepare_params method.

Benefits

• Property Ordering: Control the order of properties in structured outputs
• Advanced Constraints: Support for min_items, max_items, enum values, etc.
• Backward Compatibility: Existing dict-based schemas continue to work unchanged
• Fine-grained Control: Direct access to all OpenAPI Schema features supported by Gemini

Type

🆕 New Feature

Testing

I added a new integration test called test_structured_output_schema_json_with_openapi_schema_object.

How to use

from langchain_google_vertexai import ChatVertexAI
from google.cloud.aiplatform_v1beta1.types import Schema, Type

llm = ChatVertexAI(
    model="gemini-2.0-flash",
    temperature=0.7,
    project="YOUR_PROJECT_ID",
    location="YOUR_LOCATION",
    request_parallelism=1,
    credentials=YOUR_CREDENTIALS,
    max_retries=0,
)

schema = Schema(
    type_=Type.ARRAY,
    items=Schema(
        type_=Type.OBJECT,
        properties={
            "name": Schema(type_=Type.STRING),
            "age": Schema(type_=Type.INTEGER),
            "hobby": Schema(
                type_=Type.ARRAY, items=Schema(type_=Type.STRING), max_items=2
            ),
        },
        required=["name", "age"],
        property_ordering=["name", "age"],
    ),
    min_items=2,
)

llm.with_structured_output(schema, method="json_mode").invoke(
    "My name is Minki and I am 20 years old. I live in Montreal. My hobby is reading, writing, and coding. My wife is Alison and she is 18 years old. She likes to cook and bake."
)

[{'name': 'Minki', 'age': 20, 'hobby': ['reading', 'writing']}, {'name': 'Alison', 'age': 18, 'hobby': ['cook', 'bake']}]