|
| 1 | +# Schema Registry Example - External HTTP Server |
| 2 | + |
| 3 | +This page demonstrates how the Schema Registry and Rule Engine support message encoding and decoding using an external HTTP server with custom logic. |
| 4 | + |
| 5 | +In some scenarios, you might need to apply custom encoding or decoding logic that EMQX does not support natively. EMQX allows you to delegate this processing to an external HTTP service by invoking it through `schema_encode` and `schema_decode` functions within a rule. |
| 6 | + |
| 7 | +## External HTTP API Specification |
| 8 | + |
| 9 | +To implement a custom External HTTP API that integrates with EMQX's `schema_encode` and `schema_decode` functions, your External HTTP server must provide a single `POST` endpoint that handles the encoding or decoding requests from EMQX. |
| 10 | + |
| 11 | +### Request Format |
| 12 | + |
| 13 | +The request body is a JSON object with the following fields: |
| 14 | + |
| 15 | +- `payload`: Base64-encoded string value passed to the `schema_encode` or `schema_decode` function in Rule Engine. |
| 16 | +- `type`: Either the `encode` or the `decode` string, depending on which function is evaluated, `schema_encode` or `schema_decode`. |
| 17 | +- `schema_name`: A string identifying the name of this External HTTP schema configured in EMQX. |
| 18 | +- `opts`: An arbitrary string that can be configured in EMQX to provide further options, which is passed unaltered to the HTTP server. |
| 19 | + |
| 20 | +### Response Format |
| 21 | + |
| 22 | +- The server must respond with HTTP status code `200`. |
| 23 | +- The response body must contain a base64-encoded string representing the result. Note that this base64 value must not be further JSON-encoded when replying to EMQX. |
| 24 | + |
| 25 | +## Example Use Case |
| 26 | + |
| 27 | +Suppose a device publishes a binary message, and you want to encode or decode the payload using a custom XOR operation. This section demonstrates how to integrate custom encoding and decoding logic into EMQX by building a simple external HTTP service. |
| 28 | + |
| 29 | +### Build an External HTTP Service |
| 30 | + |
| 31 | +The following example demonstrates how to create and run a simple HTTP server using Python and Flask. The server receives Base64-encoded data and applies an XOR operation to the decoded payload. |
| 32 | + |
| 33 | +<details> |
| 34 | +<summary><strong>Code for sample External HTTP Server</strong></summary> |
| 35 | + |
| 36 | +Ensure [Flask](https://flask.palletsprojects.com/en/stable/) is installed: |
| 37 | + |
| 38 | +```sh |
| 39 | +pip install Flask==3.1.0 |
| 40 | +``` |
| 41 | + |
| 42 | +Sample code: |
| 43 | + |
| 44 | +```python |
| 45 | +from flask import Flask, request |
| 46 | +import base64 |
| 47 | + |
| 48 | +app = Flask(__name__) |
| 49 | + |
| 50 | +@app.route("/serde", methods=['POST']) |
| 51 | +def serde(): |
| 52 | + # The input payload is base64 encoded |
| 53 | + body = request.get_json(force=True) |
| 54 | + print("incoming request:", body) |
| 55 | + payload64 = body.get("payload") |
| 56 | + payload = base64.b64decode(payload64) |
| 57 | + secret = 122 |
| 58 | + response = bytes(b ^ secret for b in payload) |
| 59 | + # The response must also be base64 encoded |
| 60 | + response64 = base64.b64encode(response) |
| 61 | + return response64 |
| 62 | +``` |
| 63 | + |
| 64 | +To run your server: |
| 65 | + |
| 66 | +```sh |
| 67 | +# This assumes your server is in the same directory in a file named `myapp.py` |
| 68 | +flask --app myapp --debug run -h 0.0.0.0 -p 9500 |
| 69 | +``` |
| 70 | + |
| 71 | +</details> |
| 72 | + |
| 73 | +### Create External HTTP Schema in EMQX |
| 74 | + |
| 75 | +1. Go to the Dashboard, and select **Smart Data Hub** -> **Schema Registry** from the left navigation menu. |
| 76 | + |
| 77 | +2. In the **Internal** tab page, click **Create**. |
| 78 | + |
| 79 | +3. Create an External HTTP server schema using the following parameters: |
| 80 | + - **Name**: `myhttp` |
| 81 | + |
| 82 | + - **Type**: `External HTTP` |
| 83 | + |
| 84 | + - **URL**: The full URI where your server is running. For example: `http://server:9500/serde`. |
| 85 | + |
| 86 | +4. Click **Create**. |
| 87 | + |
| 88 | +### Create a Rule to Apply Schema |
| 89 | + |
| 90 | +Use the EMQX rule engine to create a rule that applies your schema for message encoding and decoding. |
| 91 | + |
| 92 | +1. In the Dashboard, select **Integration** -> **Rules** from the navigation menu. |
| 93 | + |
| 94 | +2. On the **Rules** page, click **Create** at the top right corner. |
| 95 | + |
| 96 | +3. Use the schema you have just created to write the rule SQL statement: |
| 97 | + |
| 98 | + ```sql |
| 99 | + SELECT |
| 100 | + schema_encode('myhttp', payload) as encoded, |
| 101 | + schema_decode('myhttp', encoded) as decoded |
| 102 | + FROM |
| 103 | + "t/external_http" |
| 104 | + ``` |
| 105 | + |
| 106 | + Both `schema_encode('myhttp', payload)` and `schema_decode('myhttp', encoded)` will call the configured External HTTP server to encode/decode the given payload. |
| 107 | + |
| 108 | +4. Click **Add Action**. Select `Republish` from the drop-down list of the **Action** field. |
| 109 | + |
| 110 | +5. In the **Topic** field, type `external_http/out` as the destination topic. |
| 111 | + |
| 112 | +6. In the **Payload** field, type message content template: `${.}`. |
| 113 | + |
| 114 | +7. Click **Add** to add the action to the rule. |
| 115 | + |
| 116 | + This action sends the decoded message to the topic `external_http/out` in JSON format. `${.}` is a variable placeholder that will be replaced at runtime with the value of the whole output of the rule. |
| 117 | + |
| 118 | +8. Click **Save** to complete the rule creation. |
| 119 | + |
| 120 | +### Check Rule Execution Results |
| 121 | + |
| 122 | +1. In the Dashboard, select **Diagnose** -> **WebSocket Client**. |
| 123 | +2. Fill in the connection information for the current EMQX instance. |
| 124 | + - If you run EMQX locally, you can use the default value. |
| 125 | + - If you have changed EMQX's default configuration. For example, the configuration change on authentication can require you to type in a username and password. |
| 126 | +3. Click **Connect** to connect to the EMQX instance as an MQTT client. |
| 127 | +4. In the **Subscription** area, type `external_http/out` in the **Topic** field and click **Subscribe**. |
| 128 | + |
| 129 | +5. In the **Publish** area, type `t/external_http` in the **Topic** field, write any payload you wish, and click **Publish**. |
| 130 | + |
| 131 | +6. Check that a message with the topic `external_http/out` is received on the Websocket side. For example, if your payload was `hello`: |
| 132 | + |
| 133 | + ```json |
| 134 | + {"encoded":"\u0012\u001F\u0016\u0016\u0015","decoded":"hello"} |
| 135 | + ``` |
0 commit comments