Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Offline sender sometimes uses a fake run "id" when adding events #720

Open
alahiff opened this issue Feb 24, 2025 · 4 comments
Open

Offline sender sometimes uses a fake run "id" when adding events #720

alahiff opened this issue Feb 24, 2025 · 4 comments
Assignees
Labels
ready for testing This issue has been fixed and is ready for testing
Milestone

Comments

@alahiff
Copy link
Collaborator

alahiff commented Feb 24, 2025

Sometimes instead of using the actual run id the offline sender uses a string in the form offline_... containing a uuid. I think this an internal placeholder used by the offline sender (?) and results in a 422 error from the server.

Sorry, I'm not sure how to reliably reproduce this.

@wk9874
Copy link
Collaborator

wk9874 commented Feb 24, 2025

Hmmm... Yeah you're correct, in offline mode a randomly generated offline ID is assigned to each object on creation. So an Event knows a Run by its offline ID initially. The sender should then keep a mapping of online and offline IDs and swap them just before the event is uploaded. Not sure why it would sometimes be failing to do that correctly, if you could find a way to reproduce it that'd be great!

@wk9874
Copy link
Collaborator

wk9874 commented Feb 24, 2025

This will probably be caused by a run being closed and so its online ID being deleted by the sender, but then trying to upload another event in a second sender iteration... Just to check @alahiff do you only have one sender running at a time?

@wk9874
Copy link
Collaborator

wk9874 commented Feb 24, 2025

Could also happen if the sender is stopped mid-way through (or failed?) and there are some remnants in the cache from previous runs (but would need to have been stopped between the runs being uploaded and their cached files deleted, and metrics/events being uploaded from the cache)

@wk9874
Copy link
Collaborator

wk9874 commented Feb 24, 2025

Ahhhh, alternatively it could be that a run and events have been created between the sender starting to upload runs, and starting to upload events. Should make it so that the sender gets its list of files to upload as the very first thing it does for all object types, not one object at a time.

@wk9874 wk9874 self-assigned this Feb 24, 2025
@wk9874 wk9874 added the in progress Work is being done to resolve this issue label Feb 24, 2025
@wk9874 wk9874 added in review This issue is resolved in a Pull Request which is being reviewed and removed in progress Work is being done to resolve this issue labels Feb 25, 2025
@wk9874 wk9874 mentioned this issue Feb 27, 2025
@wk9874 wk9874 added ready for testing This issue has been fixed and is ready for testing and removed in review This issue is resolved in a Pull Request which is being reviewed labels Feb 27, 2025
@wk9874 wk9874 assigned alahiff and unassigned wk9874 Feb 27, 2025
@wk9874 wk9874 added this to the v2.0.0-rc1 milestone Mar 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready for testing This issue has been fixed and is ready for testing
Projects
None yet
Development

No branches or pull requests

2 participants