Merge pull request #194 from williaminfante/feat/pact-message-2

feat: implement pact message

Author: elliottmurray

Makefile

@@ -40,13 +40,29 @@ define E2E
 endef
 export E2E
 
+
+define messaging
+	echo "messaging make"
+	cd examples/message
+  pip install -r requirements.txt
+  pip install -e ../../
+  pytest
+endef
+export messaging
+
+
 .PHONY: e2e
 e2e:
 	bash -c "$$E2E"
 
+
+.PHONY: messaging
+messaging:
+	bash -c "$$messaging"
+
+
 .PHONY: examples
-examples: e2e
-	
+examples: e2e messaging
 
 
 .PHONY: package

examples/message/README.md

@@ -0,0 +1,179 @@
+# Introduction
+
+This is an e2e example that uses messages, including a sample implementation of a message handler.
+
+## Consumer
+
+A Consumer is the system that will be reading a message from a queue or some intermediary. In this example, the consumer is a Lambda function that handles the message.
+
+From a Pact testing point of view, Pact takes the place of the intermediary (MQ/broker etc.) and confirms whether or not the consumer is able to handle a request.
+
+```
++-----------+          +-------------------+
+| (Pact)    | message  |(Message Consumer) |
+| MQ/broker |--------->|Lambda Function    |
+|           |          |check valid doc    |
++-----------+          +-------------------+
+```
+
+Below is a sample message handler that only accepts that the key `documentType` would only be `microsoft-word`. If not, the message handler will throw an exception (`CustomError`)
+
+```python
+class CustomError(Exception):
+    def __init__(self, *args):
+        if args:
+            self.topic = args[0]
+        else:
+            self.topic = None
+
+    def __str__(self):
+        if self.topic:
+            return 'Custom Error:, {0}'.format(self.topic)
+
+class MessageHandler(object):
+    def __init__(self, event):
+        self.pass_event(event)
+
+    @staticmethod
+    def pass_event(event):
+        if event.get('documentType') != 'microsoft-word':
+            raise CustomError("Not correct document type")
+```
+
+Below is a snippet from a test where the message handler has no error.
+Since the expected event contains a key `documentType` with value `microsoft-word`, message handler does not throw an error and a pact file `f"pacts/{expected_json}"` is expected to be generated.
+
+```python
+def test_generate_new_pact_file(pact):
+    cleanup_json(PACT_FILE)
+
+    expected_event = {
+        'documentName': 'document.doc',
+        'creator': 'TP',
+        'documentType': 'microsoft-word'
+    }
+
+    (pact
+     .given('A document create in Document Service')
+     .expects_to_receive('Description')
+     .with_content(expected_event)
+     .with_metadata({
+         'Content-Type': 'application/json'
+     }))
+
+    with pact:
+        # handler needs 'documentType' == 'microsoft-word'
+        MessageHandler(expected_event)
+
+    progressive_delay(f"{PACT_FILE}")
+    assert isfile(f"{PACT_FILE}") == 1
+```
+
+For a similar test where the event does not contain a key `documentType` with value `microsoft-word`, a `CustomError` is generated and there there is no generated json file `f"pacts/{expected_json}"`.
+
+```python
+def test_throw_exception_handler(pact):
+    cleanup_json(PACT_FILE)
+    wrong_event = {
+        'documentName': 'spreadsheet.xls',
+        'creator': 'WI',
+        'documentType': 'microsoft-excel'
+    }
+
+    (pact
+     .given('Another document in Document Service')
+     .expects_to_receive('Description')
+     .with_content(wrong_event)
+     .with_metadata({
+         'Content-Type': 'application/json'
+     }))
+
+    with pytest.raises(CustomError):
+        with pact:
+            # handler needs 'documentType' == 'microsoft-word'
+            MessageHandler(wrong_event)
+
+    progressive_delay(f"{PACT_FILE}")
+    assert isfile(f"{PACT_FILE}") == 0
+```
+
+Otherwise, no pact file is generated.
+
+## Provider
+
+Note: The current example only tests the consumer side.
+In the future, provider tests will also be included.
+
+```
++-------------------+          +-----------+
+|(Message Provider) | message  | (Pact)    |
+|Document Upload    |--------->| MQ/broker |
+|Service            |          |           |
++-------------------+          +-----------+
+```
+
+## E2E Messaging
+
+Note: The current example only tests the consumer side.
+In the future, provider tests will also be included.
+
+```
++-------------------+          +-----------+          +-------------------+
+|(Message Provider) | message  | (Pact)    | message  |(Message Consumer) |
+|Document Upload    |--------->| MQ/broker |--------->|Lambda Function    |
+|Service            |          |           |          |check valid doc    |
++-------------------+          +-----------+          +-------------------+
+```
+
+# Setup
+
+## Virtual Environment
+
+Go to the `example/message` directory Create your own virtualenv for this. Run
+
+```bash
+pip install -r requirements.txt
+pip install -e ../../
+pytest
+```
+
+## Message Consumer
+
+From the root directory run:
+
+```bash
+pytest
+```
+
+Or you can run individual tests like:
+
+```bash
+pytest tests/consumer/test_message_consumer.py::test_generate_new_pact_file
+```
+
+## With Broker
+
+The current consumer test can run even without a local broker,
+but this is added for demo purposes.
+
+Open a separate terminal in the `examples/broker` folder and run:
+
+```bash
+docker-compose up
+```
+
+Open a browser to http://localhost and see the broker you have succeeded.
+If needed, log-in using the provided details in tests such as:
+
+```
+PACT_BROKER_USERNAME = "pactbroker"
+PACT_BROKER_PASSWORD = "pactbroker"
+```
+
+To get the consumer to publish a pact to broker,
+open a new terminal in the `examples/message` and run the following (2 is an arbitary version number). The first part makes sure that the an existing json has been generated:
+
+```bash
+pytest tests/consumer/test_message_consumer.py::test_publish_to_broker
+pytest tests/consumer/test_message_consumer.py::test_publish_to_broker --publish-pact 2
+```

examples/message/requirements.txt

@@ -0,0 +1,3 @@
+Flask==1.1.1
+pytest==5.4.1
+requests==2.23.0

examples/message/run_pytest.sh

@@ -0,0 +1,7 @@
+#!/bin/bash
+set -o pipefail
+
+pytest
+
+# publish to broker assuming broker is active
+# pytest tests/consumer/test_message_consumer.py::test_publish_to_broker --publish-pact 2

examples/message/src/__init__.py

@@ -0,0 +1,19 @@
+class CustomError(Exception):
+    def __init__(self, *args):
+        if args:
+            self.topic = args[0]
+        else:
+            self.topic = None
+
+    def __str__(self):
+        if self.topic:
+            return 'Custom Error:, {0}'.format(self.topic)
+
+class MessageHandler(object):
+    def __init__(self, event):
+        self.pass_event(event)
+
+    @staticmethod
+    def pass_event(event):
+        if event.get('documentType') != 'microsoft-word':
+            raise CustomError("Not correct document type")

examples/message/src/message_handler.py

@@ -0,0 +1,6 @@
+
+def pytest_addoption(parser):
+    parser.addoption(
+        "--publish-pact", type=str, action="store",
+        help="Upload generated pact file to pact broker with version"
+    )

examples/message/tests/__init__.py

@@ -0,0 +1,142 @@
+"""pact test for a message consumer"""
+
+import logging
+import pytest
+import time
+
+from os import remove
+from os.path import isfile
+
+from pact import MessageConsumer, Provider
+from src.message_handler import MessageHandler, CustomError
+
+log = logging.getLogger(__name__)
+logging.basicConfig(level=logging.INFO)
+
+PACT_BROKER_URL = "http://localhost"
+PACT_BROKER_USERNAME = "pactbroker"
+PACT_BROKER_PASSWORD = "pactbroker"
+PACT_DIR = 'pacts'
+
+CONSUMER_NAME = 'DetectContentLambda'
+PROVIDER_NAME = 'ContentProvider'
+PACT_FILE = (f"{CONSUMER_NAME.lower().replace(' ', '_')}_message-"
+             + f"{PROVIDER_NAME.lower().replace(' ', '_')}_message.json")
+
+@pytest.fixture(scope='session')
+def pact(request):
+    version = request.config.getoption('--publish-pact')
+    publish = True if version else False
+
+    pact = MessageConsumer(CONSUMER_NAME, version=version).has_pact_with(
+        Provider(PROVIDER_NAME),
+        publish_to_broker=publish, broker_base_url=PACT_BROKER_URL,
+        broker_username=PACT_BROKER_USERNAME, broker_password=PACT_BROKER_PASSWORD)
+
+    # current pact does not consider the PACT_DIR argument, assumes none
+    yield pact
+
+
+def cleanup_json(file):
+    """
+    Remove existing json file before test if any
+    """
+    if (isfile(f"{file}")):
+        remove(f"{file}")
+
+
+def progressive_delay(file, time_to_wait=10, second_interval=0.5, verbose=False):
+    """
+    progressive delay
+    defaults to wait up to 5 seconds with 0.5 second intervals
+    """
+    time_counter = 0
+    while not isfile(file):
+        time.sleep(second_interval)
+        time_counter += 1
+        if verbose:
+            print(f'Trying for {time_counter*second_interval} seconds')
+        if time_counter > time_to_wait:
+            if verbose:
+                print(f'Already waited {time_counter*second_interval} seconds')
+            break
+
+
+def test_throw_exception_handler(pact):
+    cleanup_json(PACT_FILE)
+    wrong_event = {
+        'documentName': 'spreadsheet.xls',
+        'creator': 'WI',
+        'documentType': 'microsoft-excel'
+    }
+
+    (pact
+     .given('Another document in Document Service')
+     .expects_to_receive('Description')
+     .with_content(wrong_event)
+     .with_metadata({
+         'Content-Type': 'application/json'
+     }))
+
+    with pytest.raises(CustomError):
+        with pact:
+            # handler needs 'documentType' == 'microsoft-word'
+            MessageHandler(wrong_event)
+
+    progressive_delay(f"{PACT_FILE}")
+    assert isfile(f"{PACT_FILE}") == 0
+
+
+def test_generate_new_pact_file(pact):
+    cleanup_json(PACT_FILE)
+
+    expected_event = {
+        'documentName': 'document.doc',
+        'creator': 'TP',
+        'documentType': 'microsoft-word'
+    }
+
+    (pact
+     .given('A document create in Document Service')
+     .expects_to_receive('Description')
+     .with_content(expected_event)
+     .with_metadata({
+         'Content-Type': 'application/json'
+     }))
+
+    with pact:
+        # handler needs 'documentType' == 'microsoft-word'
+        MessageHandler(expected_event)
+
+    progressive_delay(f"{PACT_FILE}")
+    assert isfile(f"{PACT_FILE}") == 1
+
+
+def test_publish_to_broker(pact):
+    """
+    This test does not clean-up previously generated pact.
+    Sample execution where 2 is an arbitrary version:
+
+    `pytest tests/consumer/test_message_consumer.py::test_publish_pact_to_broker`
+
+    `pytest tests/consumer/test_message_consumer.py::test_publish_pact_to_broker --publish-pact 2`
+    """
+    expected_event = {
+        'documentName': 'document.doc',
+        'creator': 'TP',
+        'documentType': 'microsoft-word'
+    }
+
+    (pact
+     .given('A document create in Document Service with broker')
+     .expects_to_receive('Description with broker')
+     .with_content(expected_event)
+     .with_metadata({
+         'Content-Type': 'application/json'
+     }))
+
+    with pact:
+        MessageHandler(expected_event)
+
+    progressive_delay(f"{PACT_FILE}")
+    assert isfile(f"{PACT_FILE}") == 1