feat(AP-6075): Add publisher & consumer level message deduplication

[Drodevbar]

This PR adds publisher-level and consumer-level message deduplication mechanisms.

Changelog:

1. add redis-message-deduplication-store package as ioredis-based deduplication storage (utilizes redis-sempaphore under the hood)
2. adjust SQS & SNS publishers and SNS publisher manager to support publisher-level deduplication
3. adjust SQS consumer to support consumer-level deduplication

[kibertoad]

what if message fails with "Error, do not retry"? (e. g. payload cannot be parsed or does not pass validation).

Are we / should we mark the message as PROCESSED?

[Drodevbar]

I've addressed 2., 3. and 4. scenarios from the below list:

1. message validation fails and we remove the message from the queue
2. retryLater and message should be retried
3. retryLater and message should not be retried (due to exceeding the retries)
4. error during processing the message

In cases 2., 3. and 4. I'm releasing the lock before changing the message status to allow further processing. In the case of 1, at this stage, the lock has not yet been acquired, so we don't need to do anything specific.

Hope it makes sense 🙌

[kibertoad]

to reduce probability of errors, if (consumer)publisherDeduplication is enabled, and deduplicationId field is not present, I'd throw an error

[Drodevbar]

@kibertoad Even if publisher/consumer-level deduplication is enabled, it's still possible to skip it by omitting the deduplication ID. This is by design as our publishers support multiple message types, so if dedup is enabled on the publisher level, not all message types will most likely support it.

Previously, this was being configured per message type on the consumer/publisher/manager level, but I've adjusted it to pass it as message metadata as discussed there with Krzysztof and Eduards: #236 (comment)

Let me know if it makes sense.