Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a flush function to the Run class #590

Open
james-panayis opened this issue Nov 7, 2024 · 0 comments
Open

Add a flush function to the Run class #590

james-panayis opened this issue Nov 7, 2024 · 0 comments
Assignees

Comments

@james-panayis
Copy link
Collaborator

Sometimes the direct dispatcher is impractical to use, as it results in prohibitively many requests being made to the server. But in these cases, users may still want to be able to query metrics that they have already told a Run to upload.

For this case, we need a .flush() function which will empty any queues in the dispatcher (sending their contents to the server). Crucially, control should not flow out of the .flush() function until all those uploads have finished, so that any subsequent queries of that data are guaranteed to be correct.

The use case that has made this requirement apparent is using connectors (which upload lots of things to a run - so many that the direct dispatcher causes problems) with the optimization framework (where we often want to query the contents of a still-open run just after metrics have been sent to it).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants