While setting up the project locally and running the test data loader,
ran into a few issues that prevent any data from being ingested.
Problems found:
-
The ts field in SAMPLE_SLACK_MESSAGES is a float number but the Go
ingestion service expects it as a string. This causes a JSON unmarshal
error and every Slack webhook call returns 400.
-
Related to the above — int(message['ts']) in create_slack_event()
crashes with a ValueError because you can't directly cast a float string
like "1711097400.000001" to int. Needs int(float(...)) instead.
-
The script sends raw POST requests with no signature headers. The
ingestion service validates HMAC signatures for both Slack and GitHub
webhooks, so every request returns 401 without them.
Fix:
- Changed all ts values to strings in SAMPLE_SLACK_MESSAGES
- Fixed int(message['ts']) → int(float(message['ts']))
- Added X-Slack-Signature + X-Slack-Request-Timestamp headers for Slack
- Added X-Hub-Signature-256 + X-GitHub-Event headers for GitHub
After these fixes all 8 events load successfully.
Steps to reproduce:
Clone the repo, run docker-compose up -d, then python test_data/load_test_data.py
and observe the 401 and 400 errors.
While setting up the project locally and running the test data loader,
ran into a few issues that prevent any data from being ingested.
Problems found:
The
tsfield in SAMPLE_SLACK_MESSAGES is a float number but the Goingestion service expects it as a string. This causes a JSON unmarshal
error and every Slack webhook call returns 400.
Related to the above —
int(message['ts'])in create_slack_event()crashes with a ValueError because you can't directly cast a float string
like "1711097400.000001" to int. Needs int(float(...)) instead.
The script sends raw POST requests with no signature headers. The
ingestion service validates HMAC signatures for both Slack and GitHub
webhooks, so every request returns 401 without them.
Fix:
After these fixes all 8 events load successfully.
Steps to reproduce:
Clone the repo, run docker-compose up -d, then python test_data/load_test_data.py
and observe the 401 and 400 errors.