Skip to content

Conversation

terylt
Copy link
Collaborator

@terylt terylt commented Aug 22, 2025

✨ Feature / Enhancement PR

πŸ”— Epic / Issue

Link to the epic or parent issue:
Closes #


πŸš€ Summary

This PR does two things:

  1. It enables support for context sharing within a plugin and across plugins.
  2. It improves plugin error handling and standardizes it.

πŸ§ͺ Checks

  • make lint passes
  • make test passes
  • CHANGELOG updated (if user-facing)

πŸ““ Notes

On Errors:

Currently, plugin errors are being handled as follows:

  • Errors are caught in the plugin manager, and then bubbled up to the gateway as PluginViolation Objects, with the idea that if the plugin is in enforcement mode, it blocks the continued processing. If the plugin is in permissive mode, the error is logged, but the system is allowed to go on.

I'm do not think PluginViolation is the best way to bubble up an error. It's also not clear how this part works with the global plugin settings:

plugin_settings:
parallel_execution_within_band: true
plugin_timeout: 30
fail_on_plugin_error: false <---------------------------------- this..
enable_plugin_api: true
plugin_health_check_interval: 60

I reimplemented it a bit and would like feedback:

  1. I created a PluginError exception object.
  2. If a plugin throws an exception, the plugin manager catches it and logs it.
  3. if fail_on_plugin_error is set to true it bubbles that exception up as a PluginError regardless of the plugin mode.
  4. if fail_on_plugin_error is set to false the error is handled based off of the plugin mode as follows:
    • if PluginMode is enforce both violations and errors are bubbled up as exceptions and the execution is blocked.
    • if PluginMode is enforce_ignore_error violations are bubbled up as exceptions and execution is blocked, but errors are logged and execution continues.
    • if PluginMode is permissive execution is allowed to proceed whether there are errors or violations. Both are logged.

On Context Sharing

Each plugin hook takes a PluginContext. The PluginContext now looks as follows:

class GlobalContext(BaseModel):
    """The global context, which shared across all plugins.

    Attributes:
            request_id (str): ID of the HTTP request.
            user (str): user ID associated with the request.
            tenant_id (str): tenant ID.
            server_id (str): server ID.
            metadata (Optional[dict[str,Any]]): a global shared metadata across plugins.
            state (Optional[dict[str,Any]]): a global shared state across plugins.

    Examples:
        >>> ctx = GlobalContext(request_id="req-123")
        >>> ctx.request_id
        'req-123'
        >>> ctx.user is None
        True
        >>> ctx2 = GlobalContext(request_id="req-456", user="alice", tenant_id="tenant1")
        >>> ctx2.user
        'alice'
        >>> ctx2.tenant_id
        'tenant1'
        >>> c = GlobalContext(request_id="123", server_id="srv1")
        >>> c.request_id
        '123'
        >>> c.server_id
        'srv1'
    """

    request_id: str
    user: Optional[str] = None
    tenant_id: Optional[str] = None
    server_id: Optional[str] = None
    state: dict[str, Any] = {}
    metadata: dict[str, Any] = {}


class PluginContext(BaseModel):
    """The plugin's context, which lasts a request lifecycle.

    Attributes:
       state:  the inmemory state of the request.
       global_context: the context that is shared across plugins.
       metadata: plugin meta data.

    Examples:
        >>> gctx = GlobalContext(request_id="req-123")
        >>> ctx = PluginContext(global_context=gctx)
        >>> ctx.global_context.request_id
        'req-123'
        >>> ctx.global_context.user is None
        True
        >>> ctx.state["somekey"] = "some value"
        >>> ctx.state["somekey"]
        'some value'
    """

    state: dict[str, Any] = {}
    global_context: GlobalContext
    metadata: dict[str, Any] = {}

The GlobalContext is designed to store global information about the hookpoint. Items such as:

  1. request id
  2. server id
  3. tenant id
  4. Tool, prompt, resource information - stored as metadata.

Plugins can pass state information to OTHER plugins by setting the values in the GlobalContext->state object. State information is maintained across pre/post hooks (for a tool for example) but is deleted after a pre/post hook pair is complete.

The global context is accessible through the PluginContext using the global_context field. This field represents a copy of the GlobalContext and is updated during each plugin pre and post hook invocation.

Plugins can pass state information across PRE/POST hook pairs within the SAME plugin using the PluginContext->state object. Data is deleted from the state after pre/post invocations.

PluginContext->metadata is designed to hold PluginContext statistical information. This could be removed in a later release if not deemed useful.

@terylt terylt force-pushed the feat/shared_plugin_context branch from d14876f to fa1d735 Compare August 26, 2025 17:02
@terylt terylt marked this pull request as ready for review August 27, 2025 14:10
@araujof araujof self-requested a review August 28, 2025 00:01
Copy link
Member

@araujof araujof left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work! Overall this is a solid step toward predictable cross-plugin state sharing and clearer failure modes.

This PR introduces:

  1. a shared GlobalContext that flows across plugins within a request, alongside a per-plugin PluginContext; and
  2. a rework of error handling semantics, adding enforce_ignore_error and wiring behavior to fail_on_plugin_error.

The server now returns a structured result object {result, context?, error?}. New unit tests and fixtures were added to exercise context propagation and error modes.

I like the design! Please consider the following:

  • Fix the Pydantic mutable defaults
  • Consider the small change to add the plugin_name to the server result, consider the additional check on the global context metadata
  • Add a CHANGELOG note for the new mode + server response shape
  • Document the new enforce_ignore_error mode

try:
if plugin:
_payload = payload_model.model_validate(payload)
_context = PluginContext.model_validate(context)
result = await asyncio.wait_for(hook_function(plugin, _payload, _context), plugin_timeout)
return result.model_dump()
result_payload[RESULT] = result.model_dump()
if _context.state or _context.metadata or _context.global_context.state:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe consider wrapping this expression into an instance method is_empty(self) -> bool.
e.g., if _context.is_empty(): ...

Also, do we need to include _context.global_context.metadata in the expression?

@@ -662,17 +668,32 @@ class GlobalContext(BaseModel):
user: Optional[str] = None
tenant_id: Optional[str] = None
server_id: Optional[str] = None
state: dict[str, Any] = {}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think having mutable defaults ({}) assign this way can be an issue; the dict is created once at class definition time and may be shared across instances. Use Field(default_factory=dict) instead. This could cause subtle state bleed across requests/plugins (didn't verify it).

state: dict[str, Any] = {}
metadata: dict[str, Any] = {}

Replace = {} with Field(default_factory=dict) for all dict fields (state, metadata), and consider the same for any lists/sets.

True
>>> ctx.state["somekey"] = "some value"
>>> ctx.state["somekey"]
'some value'
"""

state: dict[str, Any] = {}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar issue here (state, metadata) with mutable dict assignments. Change to default_factory.

False
"""
global_plugin_manager = PluginManager()
plugin_timeout = global_plugin_manager.config.plugin_settings.plugin_timeout if global_plugin_manager.config else DEFAULT_PLUGIN_TIMEOUT
plugin = global_plugin_manager.get_plugin(plugin_name)
result_payload: dict[str, Any] = {}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about we do this?

- result_payload: dict[str, Any] = {}
+ result_payload: dict[str, Any] = {"plugin": plugin_name}

This would always return the plugin_name in the result, which could be handy for logging and other things without extra lookups.

@araujof araujof added the enhancement New feature or request label Aug 28, 2025
@araujof araujof added this to the Release 0.7.0 milestone Aug 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants