Test and Improve Relay Automation Provider Realism

[RAWx18]

Context

LynxCapital is a simulated company used to create realistic environments for testing Caracal. Instead of using real providers that require credentials, we build provider implementations that mimic real-world services and expose them over the network through containers. LynxCapital consumes these providers exactly like it would consume external third-party services.

Relay Automation is the workflow and job automation provider used by LynxCapital to dispatch long-running operations, orchestrate workflows, manage job execution, and coordinate tool-driven automation processes.

This provider exposes capabilities through MCP and uses Caracal Mandate authentication. The provider should be treated as a Caracal ecosystem partner service that has adopted Caracal Mandate authorization and uses the verifier implementation from the Caracal SDK to validate mandates before executing automation workflows and jobs.

Task

Review and improve the Relay Automation provider to ensure it reflects a realistic workflow automation platform operating within the Caracal ecosystem.

Focus on validating that the provider behaves like an independent partner service using MCP and Caracal Mandate rather than a provider designed specifically around testing scenarios.

Review and improve:

• MCP server design
• Tool definitions and naming conventions
• Caracal Mandate verification flow
• SDK verifier integration
• Authorization behavior
• Workflow execution patterns
• Long-running job handling
• Job state management
• Request and response schemas
• Error handling patterns
• Audit and execution tracking

Expected Work

• Review MCP implementation and tool definitions
• Review Caracal Mandate verification implementation
• Validate proper usage of the Caracal SDK verifier
• Review all tools, schemas, and naming conventions
• Add missing fields commonly found in workflow and automation platforms
• Improve mock workflow, job, execution, queue, and result data
• Expand realistic success, failure, and edge-case scenarios
• Update the provider dashboard if needed
• Verify LynxCapital is properly using the provider and its data
• Improve integrations where functionality is underutilized
• Ensure execution history, auditability, and workflow traceability are realistically represented

Acceptance Criteria

• Provider resembles a realistic workflow and job automation platform
• MCP implementation follows realistic patterns
• Caracal Mandate verification is implemented correctly
• Caracal SDK verifier is integrated and used appropriately
• Tool definitions and schemas follow realistic conventions
• Long-running job behavior is realistically represented
• Mock data is realistic and useful
• Additional real-world scenarios are covered
• Dashboard remains consistent with provider capabilities
• LynxCapital properly utilizes the provider
• Audit and execution tracking workflows are realistically represented
• No changes are made solely to fit Caracal-specific behavior