Skip to content

Version 2.9.0
Compare
Choose a tag to compare
@github-actions github-actions released this 11 Apr 11:39
· 85 commits to master since this release
2.9.0
d9ee385

The primary reason for this release is to update the logic for /sink-health endpoint. It also comes with other nice improvements.

What's new ?

Improved logic for /sink-health endpoint (#276)

Until this release, the collector was starting with /sink-health endpoint healthy, until proven unhealthy (which was a problem in case of Kinesis outage for instance). The new logic is to start unhealthy until proven healthy, either by successfully sending events to the queue or by successfully checking the stream with AWS/GCP API, whichever comes first.

More details can be found on the related issue.

Kinesis and SQS: MaxRetries configurable (#295)

MaxRetries is used in 2 scenarios:

  • In case of SQS collector or Kinesis collector without SQS buffer, /sink-health becomes unhealthy after this number of failed retries (even though it is retried forever).
  • In case of Kinesis collector with SQS buffer, this is the number of retries to Kinesis before switching to SQS.

It was hard coded to 3 and is now configurable (here for Kinesis and here for SQS).

Pub/Sub: Pub/Sub emulator support (#270)

It is now possible to use Pub/Sub emulator (e.g. for testing).

Using it is as simple as running the Docker image and setting PUBSUB_EMULATOR_HOST.

Pub/Sub: second layer of retry (#304)

There is now a second layer of retry for inserting records into Pub/Sub, in addition to the built-in retry mechanism of Pub/Sub publisher. When a record can't be inserted after the built-in retries, it's added to a buffer and every retryInterval, the collector retries to insert the failed inserts.

MaxBytes configurable (#272)

In case a collector payload is too big, a size violation bad row is emitted.

The maximum size for each type of queue was hard coded (1 MB for Kinesis, 192 KB for SQS, 10 MB for Pub/Sub 1 MB for Kafka).

It is now configurable (Kinesis, SQS, Pub/Sub, Kafka).

CHANGELOG

  • Bump protobuf-java to 3.21.7 (#308)
  • PubSub: add second layer of retry (#304)
  • Replace set-output in Github Actions (#305)
  • Make MaxRetries configurable (#295)
  • Update /sink-health endpoint logic (#276)
  • Integration tests should work with both regular and distroless project (#301)
  • Scan Docker images with Snyk container monitor in deploy.yml (#296)
  • Add integration tests for Set-Cookie (#287)
  • Add integration test for doNotTrackCookie (#289)
  • Add integration test for X-Forwarded-For (#288)
  • Add integration test for custom paths (#286)
  • Add integration test for /health endpoint (#285)
  • Kinesis: add integration tests (#283)
  • Validate cookie.fallbackDomain config option on startup (#278)
  • PubSub: add integration tests (#274)
  • PubSub: make it possible to use PubSub emulator (#270)
  • Put MaxBytes in default application.conf instead of hard coding it (#272)
Assets 8
Loading