refactor: enable plugins extensibility

[terylt]

Plugin Framework Refactor: Dynamic Hook Discovery

This PR refactors the plugin framework to support flexible, dynamic hook registration patterns while maintaining
backward compatibility.

Key Changes

1. Plugin Base Class

• Plugin is now an abstract base class (ABC)
• No longer requires predefined hook methods
• Plugins only need to implement the hooks they actually use

2. Three Hook Registration Patterns

Pattern 1: Convention-Based (Simplest)

class MyPlugin(Plugin):
    async def tool_pre_invoke(self, payload, context):
        # Method name matches hook type - automatically discovered
        return ToolPreInvokeResult(continue_processing=True)

Pattern 2: Decorator-Based (Custom Names)

class MyPlugin(Plugin):
    @hook(ToolHookType.TOOL_POST_INVOKE)
    async def my_custom_method_name(self, payload, context):
        # Method name doesn't match, but decorator registers it
        return ToolPostInvokeResult(continue_processing=True)

Pattern 3: Custom Hooks (Advanced)

class MyPlugin(Plugin):
    @hook("email_pre_send", EmailPayload, EmailResult)
    async def validate_email(self, payload, context):
        # Completely new hook type with custom payload/result
        return EmailResult(continue_processing=True)

3. Dynamic Hook Discovery

The PluginManager now discovers hooks dynamically using:

1. Convention-based lookup: Checks for methods matching hook type names
2. Decorator scanning: Uses inspect.getmembers() to find @hook decorated methods
3. Signature validation: Validates parameter count and async requirement at load time

4. Hook Invocation

# Initialize plugin manager with config
manager = PluginManager("plugins/config.yaml")
await manager.initialize()

# Invoke a hook across all registered plugins
result, contexts = await manager.invoke_hook(
    ToolHookType.TOOL_PRE_INVOKE,
    payload,
    global_context=global_context
)
# Result contains aggregated output from all plugins
# contexts preserves plugin-specific state across pre/post hooks

5. Result Type System

All hooks return PluginResult[PayloadType]:

• ToolPreInvokeResult = PluginResult[ToolPreInvokePayload]
• ToolPostInvokeResult = PluginResult[ToolPostInvokePayload]
• Results can include modified_payload, metadata, violations, or just continue_processing

6. Agent Hooks

Added new hooks for intercepting and transforming agent interactions in multi-agent workflows:

agent_pre_invoke - Intercept agent requests before invocation

class AgentPreInvokePayload(PluginPayload):
    agent_id: str                           # Agent identifier (can be modified for routing)
    messages: List[Message]                 # Conversation messages (can be filtered/transformed)
    tools: Optional[List[str]] = None       # Available tools list
    headers: Optional[HttpHeaderPayload]    # HTTP headers
    model: Optional[str] = None             # Model override
    system_prompt: Optional[str] = None     # System instructions
    parameters: Optional[Dict[str, Any]]    # LLM parameters (temperature, max_tokens, etc.)

Use cases:

• Filter/transform conversation messages (content moderation, PII redaction)
• Modify agent routing (load balancing, A/B testing)
• Validate tool access and permissions
• Override model selection or system prompts

agent_post_invoke - Process agent responses after invocation

  class AgentPostInvokePayload(PluginPayload):
      agent_id: str                           # Agent identifier
      messages: List[Message]                 # Response messages from agent (can be filtered)
      tool_calls: Optional[List[Dict]]        # Tool invocations made by agent

Use cases:

• Filter/transform response messages (safety checks, post-processing)
• Audit tool invocations made by the agent
• Track conversation quality and metrics
• Block inappropriate responses

Example Usage:

  class MessageFilterPlugin(Plugin):
      async def agent_pre_invoke(
          self, 
          payload: AgentPreInvokePayload, 
          context: PluginContext
      ) -> AgentPreInvokeResult:
          """Filter messages containing blocked words."""
          blocked_words = self.config.config.get("blocked_words", [])

          # Filter out messages with blocked content
          filtered_messages = [
              msg for msg in payload.messages
              if not any(word in msg.content.text.lower() for word in blocked_words)
          ]

          if not filtered_messages:
              return AgentPreInvokeResult(
                  continue_processing=False,
                  violation=PluginViolation(
                      code="BLOCKED_CONTENT",
                      reason="All messages contained blocked content"
                  )
              )

          # Return modified payload with filtered messages
          modified_payload = AgentPreInvokePayload(
              agent_id=payload.agent_id,
              messages=filtered_messages,
              tools=payload.tools
          )
          return AgentPreInvokeResult(modified_payload=modified_payload)

These hooks enable sophisticated multi-agent orchestration patterns like message filtering for safety, conversation
routing based on content, tool access control, and cross-agent observability. See
tests/unit/mcpgateway/plugins/agent/test_agent_plugins.py for complete examples including content filtering, context
persistence across pre/post hooks, and partial message filtering.

Benefits

• Flexibility: Choose the pattern that fits your use case
• Extensibility: Create custom hooks without modifying the framework
• Type Safety: Full type hint support with validation
• Backward Compatible: Existing plugins continue to work
• Developer Experience: Clear errors when hooks are misconfigured

Testing

• Added comprehensive test suite: test_hook_patterns.py
• Demonstrates all three patterns with working examples
• All existing plugin tests pass

Documentation

• Updated plugins/README.md with detailed examples of all patterns
• Added hook signature requirements and type system explanation
• Included troubleshooting section for common issues

Files Changed

• mcpgateway/plugins/framework/base.py - Plugin ABC, dynamic hook discovery
• mcpgateway/plugins/framework/decorator.py - @hook decorator implementation
• mcpgateway/plugins/framework/hooks/*.py - Updated hook type definitions
• tests/unit/mcpgateway/plugins/fixtures/plugins/simple.py - Test fixture plugins
• plugins/README.md - Comprehensive documentation
• Fixed import paths across framework (hooks.registry not hook_registry)

Migration Guide

Existing plugins continue to work unchanged. To adopt new patterns:

1. No changes needed for plugins using convention-based naming
2. Add @hook decorator if you want custom method names
3. Define custom payloads if creating new hook types

See tests/unit/mcpgateway/plugins/framework/hooks/test_hook_patterns.py for complete working examples.

[araujof]

LGTM

This PR is a major refactoring to make the plugin system more extensible with dynamic hook discovery. Overall, this is a significant improvement for the plugin framework.

The PR maintains backward compatibility - existing plugins work unchanged.

Notes:

• I fixed lint issues, tests, and doctests. Made minor changes all around.

Summary of Changes:

• Decouples plugin framework from common gateway objects; the plugin framework is ready to be split as an independent package
• Converts Plugin to an abstract base class (ABC) for better type safety
• Introduces 3 hook registration patterns: convention-based, decorator-based, and custom hooks
• Implements dynamic hook discovery using decorators, naming conventions, and explicit registration
• Adds new agent lifecycle hooks (agent_pre_invoke, agent_post_invoke) for multi-agent workflow interception
• Standardizes all hooks to return PluginResult[PayloadType] for consistent result handling.

[araujof]

Fixed the remaining tests and linting issues. Ready to merge.