Message broker requirements

guidelines/message-broker.md

@@ -0,0 +1,139 @@
+# MONAI Deploy Message Broker Requirements/Design
+
+## Background
+
+The MONAI Deploy sub-systems, including the Workflow Manager and the Informatics Gateway, use shared virtual storage for sharing data.  With the given design, a message broker service must notify all sub-systems when data is available or when an event has occurred.
+
+This document describes and decides on a **default** message broker service to be used in the MONAI Deploy project and defines message topics/events used for publishing and subscribing events.
+
+**Important**: users shall be able to replace the **default** message broker service with their choice, and the design shall consider both on-prem, cloud, and edge deployments.
+
+##  Message Broker Services
+
+- [RabbitMQ](https://www.rabbitmq.com/) (default)
+- [KubeMQ Server](https://kubemq.io/)
+- [Google Cloud Pub/Sub](https://cloud.google.com/pubsub/)
+- [Apache Kafka](https://kafka.apache.org/intro)
+
+## Topics
+
+This section defines topics each sub-system publishes in MONAI Deploy.
+
+### Informatics Gateway
+
+* WorkflowRequest (workflow.request)
+* ExportTaskCompleted (export.completed)
+
+### Workflow Manager
+
+* ExportTaskRequest (export.request)
+* TaskRequest (task.request)
+* WorkflowStarted (workflow.started)
+* WorkflowCompleted (workflow.completed)
+
+### Task Manager (Part of Workflow Manager)
+
+* TaskStarted (task.started)
+* TaskCompleted (task.completed)
+
+## Topic Payload Specs
+
+This section defines the detailed payload of each topic.
+
+### WorkflowRequest
+| Field | Type | Description |
+|-|-|-|
+| Transaction ID | string or UUID | a unique ID to locate the workflow.  Used and generated by the event publishers. |
+| Correlation ID | string or UUID | a unique ID that traces back to the original request, e.g., a DICOM association |
+| Files | string[] | a list of file paths (URN) associated with the request |
+| Metadata | string[] | JSON formatted strings associated with the request. E.g., DICOM headers |
+| Workflows | string[] | *optional* - a list of workflows to execute |
+
+### ExportTaskCompleted
+
+| Field | Type | Description |
+|-|-|-|
+| Export Task ID | string or UUID | a unique ID to locate the export task.  Generated by the Workflow Manager. |
+| Correlation ID | string or UUID | a unique ID that traces back to the original request, e.g., a DICOM association. |
+| Transaction ID | string or UUID | the unique ID of the actual workflow request.  |
+| Status | int (enum )| Status of the export task |
+| Message | string | status details |
+| Sink | string | Sink assigned for the export task |
+
+#### ExportTask
+
+| Field | Type | Description |
+|-|-|-|
+| Export Task ID | string or UUID | a unique ID to locate the export task.  Generated by the Workflow Manager. |
+| Correlation ID | string or UUID | a unique ID that traces back to the original request, e.g., a DICOM association. |
+| Transaction ID | string or UUID | the unique ID of the actual workflow request.  |
+| Files | string[] | a list of file paths (URN) associated with the request |
+| Sink | string | Sink assigned for the export task |
+
+#### TaskRequest
+
+| Field | Type | Description |
+|-|-|-|
+| Transaction ID | string or UUID | a unique ID to locate the workflow.  Used and generated by the event publishers. |
+| Correlation ID | string or UUID | a unique ID that traces back to the original request, e.g., a DICOM association |
+| Payload Path | string | URN to the task payload |
+
+TBD
+
+#### WorkflowStarted
+
+| Field | Type | Description |
+|-|-|-|
+|-|-|-|
+| Transaction ID | string or UUID | a unique ID to locate the workflow.  Used and generated by the event publishers. |
+| Correlation ID | string or UUID | a unique ID that traces back to the original request, e.g., a DICOM association |
+| Status | int (enum) | current status of the workflow |
+| State | int (enum) | current state of the workflow |
+| DateStarted | datetime | Date time the workflow started |
+
+TBD
+
+#### WorkflowCompleted
+
+| Field | Type | Description |
+|-|-|-|
+|-|-|-|
+| Transaction ID | string or UUID | a unique ID to locate the workflow.  Used and generated by the event publishers. |
+| Correlation ID | string or UUID | a unique ID that traces back to the original request, e.g., a DICOM association |
+| Status | int (enum) | current status of the workflow |
+| State | int (enum) | current state of the workflow |
+| DateStarted | datetime | Date time the workflow started |
+| DateCompleted | datetime | Date time the workflow completed |
+
+
+TBD
+
+#### TaskStarted
+
+| Field | Type | Description |
+|-|-|-|
+| Transaction ID | string or UUID | a unique ID to locate the workflow.  Used and generated by the event publishers. |
+| Correlation ID | string or UUID | a unique ID that traces back to the original request, e.g., a DICOM association |
+| Payload Path | string | URN to the task payload |
+| Status | int (enum) | current status of the task |
+| State | int (enum) | current state of the task |
+| DateStarted | datetime | Date time the task started |
+
+
+TBD
+
+#### TaskCompleted
+
+| Field | Type | Description |
+|-|-|-|
+| Transaction ID | string or UUID | a unique ID to locate the workflow.  Used and generated by the event publishers. |
+| Correlation ID | string or UUID | a unique ID that traces back to the original request, e.g., a DICOM association |
+| Payload Path | string | URN to the task payload |
+| Status | int (enum) | current status of the task |
+| State | int (enum) | current state of the task |
+| DateStarted | datetime | Date time the task started |
+| DateCompleted | datetime | Date time the task completed |
+
+
+
+TBD