Skip to content

[WIP] Flow cancellation token to PendingItem #11196

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: dev
Choose a base branch
from

Conversation

liliankasem
Copy link
Member

Issue describing the changes in this PR

resolves #issue_for_this_pr

Pull request checklist

IMPORTANT: Currently, changes must be backported to the in-proc branch to be included in Core Tools and non-Flex deployments.

  • Backporting to the in-proc branch is not required
    • Otherwise: Link to backporting PR
  • My changes do not require documentation changes
    • Otherwise: Documentation issue linked to PR
  • My changes should not be added to the release notes for the next release
    • Otherwise: I've added my notes to release_notes.md
  • My changes do not need to be backported to a previous version
    • Otherwise: Backport tracked by issue/PR #issue_or_pr
  • My changes do not require diagnostic events changes
    • Otherwise: I have added/updated all related diagnostic events and their documentation (Documentation issue linked to PR)
  • I have added all required tests (Unit tests, E2E tests)

Additional information

Whilst investigating the core tools ctrl+c issue I was able to narrow down the problem to this area of the code.

Cancellation tokens are not passed through and so when we hit Ctrl+C and trigger the application stopping lifetime event, nothing happens because we are either waiting for a response from the worker (which never happens in this state since we kill the channel), or until timeout. The timeout is set to 1 minute and that's why it often takes a minute for the host to shut down (if the host is stopped during startup).

IRpcWorkerChannel rpcWorkerChannel = null;
string workerId = Guid.NewGuid().ToString();
_logger.LogDebug("Creating language worker channel for runtime:{runtime}", runtime);
_logger.LogWarning("Creating language worker channel for runtime:{runtime}", runtime);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

revert

@@ -89,7 +92,7 @@ public async Task<FunctionMetadataResult> GetFunctionMetadataAsync(IEnumerable<R
if (IsJobHostStarting())
{
_logger.LogDebug("JobHost is starting with state '{State}'. Initializing worker channel.", _scriptHostManager.State);
await _channelManager.InitializeChannelAsync(workerConfigs, _workerRuntime);
await _channelManager.InitializeChannelAsync(workerConfigs, _workerRuntime, _applicationLifetime.ApplicationStopping);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So my idea was to pass the _applicationLifetime.ApplicationStopping CT to StartWorkerProcessAsync -> PendingItem. (it doesn't have to come through WorkerFunctionMetadataProvider so let me know if you have better ideas)

The problem I am seeing with this is that when I cancel (exit the host) we don't see CT triggered until after the timeout period of PendingItem has run its course. We are blocking in a weird way and we need to get around that block for the CT to flow properly.

cc: @jviau @fabiocav

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant