implement distributed resizing #195

travisghansen · 2022-02-26T17:18:01Z

Signed-off-by: Travis Glenn Hansen [email protected]

What type of PR is this?

/kind feature

What this PR does / why we need it:

Add resizing support to distributed setups to compliement provisioner and snapshotter features of the same nature.

Added a parameter "--node-deployment" as a command line option for resizer sidecar, which should be set to true when the sidecar is being deployed on a per node basis.
For these changes to work, NODE_NAME environment variable must also be set while deploying the sidecar controller.

Which issue(s) this PR fixes:

Fixes #142

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

Adds support for distributed resizing.

Signed-off-by: Travis Glenn Hansen <[email protected]>

k8s-ci-robot · 2022-02-26T17:18:08Z

Welcome @travisghansen!

It looks like this is your first PR to kubernetes-csi/external-resizer 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-csi/external-resizer has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

k8s-ci-robot · 2022-02-26T17:18:09Z

Hi @travisghansen. Thanks for your PR.

I'm waiting for a kubernetes-csi member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Signed-off-by: Travis Glenn Hansen <[email protected]>

xing-yang · 2022-02-28T13:18:34Z

/ok-to-test

pohly · 2022-03-01T16:25:23Z

cmd/csi-resizer/main.go

 		informerFactory,
-		driverName)
+		driverName,
+		*enableNodeDeployment)


There is one potential scalability problem here: if there are now as many resizers running in the cluster as there are nodes, then the apiserver has to keep all of them informed about PV and PVC updates.

external-provisioner has the same problem, but it cannot really be avoided there because support for immediate provisioning depends on all provisioners seeing a new PVC.

I'm just wondering whether something better can be done for the other sidecars. The code below already checks for the util.VolumeSelectedNodeKey label. It would be possible to set up the PVC informer so that it does server-side filtering. It's doable, I just don't have a code example at hand.

That leaves the PV, though. external-provisioner would have to be modified to set a similar (the same?) label there. Such a label may even be useful for the provisioner. This code here does client-side filtering of PVs:
https://github.com/kubernetes-csi/external-provisioner/blob/3b752c36ca71fbf5d7310bb6a568b93844062189/pkg/controller/controller.go#L1132-L1145

That could be replaced with server-side filtering.

Here is code which sets up an informer with server-side filtering:
https://github.com/kubernetes-csi/external-provisioner/blob/3b752c36ca71fbf5d7310bb6a568b93844062189/cmd/csi-provisioner/csi-provisioner.go#L452-L464

I thought about that as well but didn't want to 'kick against the pricks' as they say with how the others were implemented. I also noted that we are not currently doing server-side filtering on the driver either (which I found odd).

Given my limited golang skill set I would definitely need hand-holding on proper code to manage the watches/filters should we go that route.

k8s-triage-robot · 2022-05-30T17:20:17Z

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue or PR as fresh with /remove-lifecycle stale
Mark this issue or PR as rotten with /lifecycle rotten
Close this issue or PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

travisghansen · 2022-05-30T17:33:09Z

/remove-lifecycle stale

mrpre · 2022-06-14T03:04:21Z

Any update about this pr? I patched it and it seems the resizer works well.

travisghansen · 2022-06-14T03:07:49Z

I haven’t had a chance to address the scalability concerns yet unfortunately. It should work fine for small clusters for sure.

k8s-triage-robot · 2022-09-12T04:02:12Z

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue or PR as fresh with /remove-lifecycle stale
Mark this issue or PR as rotten with /lifecycle rotten
Close this issue or PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

travisghansen · 2022-09-12T04:04:47Z

/remove-lifecycle stale

b8kings0ga · 2022-10-20T10:30:13Z

wait to use this feature

k8s-triage-robot · 2023-01-18T10:47:18Z

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue or PR as fresh with /remove-lifecycle stale
Mark this issue or PR as rotten with /lifecycle rotten
Close this issue or PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

gnufied · 2024-11-14T20:32:28Z

Why couldn't we accomplish similar thing using NodeExpand RPC calls on nodes? If there are any limitations - it would be nice to know. It is perfectly fine to have volume drivers that implement node-only NodeExpand RPC calls.

k8s-triage-robot · 2025-02-12T21:06:57Z

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

Mark this PR as fresh with /remove-lifecycle stale
Close this PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot · 2025-03-14T21:10:03Z

The Kubernetes project currently lacks enough active contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

Mark this PR as fresh with /remove-lifecycle rotten
Close this PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

travisghansen · 2025-03-22T22:24:48Z

/remove-lifecycle stale

k8s-triage-robot · 2025-04-21T22:49:25Z

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

Reopen this PR with /reopen
Mark this PR as fresh with /remove-lifecycle rotten
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

k8s-ci-robot · 2025-04-21T22:49:28Z

@k8s-triage-robot: Closed this PR.

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied

After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied

After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

Reopen this PR with /reopen

Mark this PR as fresh with /remove-lifecycle rotten

Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

travisghansen · 2025-04-21T23:08:50Z

/reopen

k8s-ci-robot · 2025-04-21T23:08:53Z

@travisghansen: Reopened this PR.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-ci-robot · 2025-04-21T23:08:59Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: travisghansen
Once this PR has been reviewed and has the lgtm label, please assign saad-ali for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-triage-robot · 2025-05-21T23:12:52Z

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

Reopen this PR with /reopen
Mark this PR as fresh with /remove-lifecycle rotten
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

k8s-ci-robot · 2025-05-21T23:12:56Z

@k8s-triage-robot: Closed this PR.

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied

After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied

After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

Reopen this PR with /reopen

Mark this PR as fresh with /remove-lifecycle rotten

Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

travisghansen · 2025-05-29T13:13:46Z

/reopen

k8s-ci-robot · 2025-05-29T13:13:50Z

@travisghansen: Reopened this PR.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-triage-robot · 2025-06-28T13:42:51Z

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

Reopen this PR with /reopen
Mark this PR as fresh with /remove-lifecycle rotten
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

k8s-ci-robot · 2025-06-28T13:42:54Z

@k8s-triage-robot: Closed this PR.

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied

After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied

After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

Reopen this PR with /reopen

Mark this PR as fresh with /remove-lifecycle rotten

Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

travisghansen · 2025-11-20T21:16:54Z

/reopen

k8s-ci-robot · 2025-11-20T21:16:57Z

@travisghansen: Reopened this PR.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

travisghansen · 2025-11-22T02:42:23Z

@gnufied can we just get this over the hill. I understand there are scaling concerns but not everyone is running clusters with 10k nodes. Let’s just get it merged and if/when scaling really becomes a problem we sort out server-side filters that can make the topology of the node.

I have thought more about simply using the controller expand operation and decided it’s not enough. Controller expand doesn’t include the same parameters (namely where the volume is mounted etc) so it’s not really feasible to do proper expansion logic.

travisghansen · 2025-11-22T02:46:37Z

Why couldn't we accomplish similar thing using NodeExpand RPC calls on nodes? If there are any limitations - it would be nice to know. It is perfectly fine to have volume drivers that implement node-only NodeExpand RPC calls.

I think node expand is already invoked and works no? I suppose you’re suggesting that node expand invoke the controller bits as a sub operation?

travisghansen · 2025-11-22T07:10:19Z

I've done some research and 1 problem is that controller requires capacity_range and node does not. The csi test suite doesn't pass capacity_range, not sure what k8s normally does though.

implement distributed resizing

91e6ac7

Signed-off-by: Travis Glenn Hansen <[email protected]>

k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. kind/feature Categorizes issue or PR as related to a new feature. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Feb 26, 2022

k8s-ci-robot requested review from gnufied and humblec February 26, 2022 17:18

k8s-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Feb 26, 2022

k8s-ci-robot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Feb 26, 2022

travisghansen mentioned this pull request Feb 26, 2022

zfs-local (non ephemereal) a possibility (possibly a feature request) democratic-csi/democratic-csi#148

Open

readme typos

b3e94c3

Signed-off-by: Travis Glenn Hansen <[email protected]>

k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Feb 28, 2022

pohly reviewed Mar 1, 2022

View reviewed changes

travisghansen mentioned this pull request Mar 28, 2022

Consolidate and improve Nomad-related documentation democratic-csi/democratic-csi#168

Closed

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label May 30, 2022

k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label May 30, 2022

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Sep 12, 2022

k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Sep 12, 2022

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jan 18, 2023

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Feb 12, 2025

k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Mar 14, 2025

k8s-ci-robot closed this Apr 21, 2025

k8s-ci-robot reopened this Apr 21, 2025

k8s-ci-robot closed this May 21, 2025

k8s-ci-robot reopened this May 29, 2025

k8s-ci-robot closed this Jun 28, 2025

k8s-ci-robot reopened this Nov 20, 2025

travisghansen mentioned this pull request Nov 20, 2025

zfs-local-dataset volume expansion democratic-csi/democratic-csi#524

Closed

implement distributed resizing #195

Are you sure you want to change the base?

implement distributed resizing #195

Conversation

travisghansen commented Feb 26, 2022

Uh oh!

k8s-ci-robot commented Feb 26, 2022

Uh oh!

k8s-ci-robot commented Feb 26, 2022

Uh oh!

xing-yang commented Feb 28, 2022

Uh oh!

pohly Mar 1, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pohly Mar 1, 2022

Choose a reason for hiding this comment

Uh oh!

travisghansen Mar 1, 2022

Choose a reason for hiding this comment

Uh oh!

k8s-triage-robot commented May 30, 2022

Uh oh!

travisghansen commented May 30, 2022

Uh oh!

mrpre commented Jun 14, 2022

Uh oh!

travisghansen commented Jun 14, 2022

Uh oh!

k8s-triage-robot commented Sep 12, 2022

Uh oh!

travisghansen commented Sep 12, 2022

Uh oh!

b8kings0ga commented Oct 20, 2022

Uh oh!

k8s-triage-robot commented Jan 18, 2023

Uh oh!

gnufied commented Nov 14, 2024

Uh oh!

k8s-triage-robot commented Feb 12, 2025

Uh oh!

k8s-triage-robot commented Mar 14, 2025

Uh oh!

travisghansen commented Mar 22, 2025

Uh oh!

k8s-triage-robot commented Apr 21, 2025

Uh oh!

k8s-ci-robot commented Apr 21, 2025

Uh oh!

travisghansen commented Apr 21, 2025

Uh oh!

k8s-ci-robot commented Apr 21, 2025

Uh oh!

k8s-ci-robot commented Apr 21, 2025

Uh oh!

k8s-triage-robot commented May 21, 2025

Uh oh!

k8s-ci-robot commented May 21, 2025

Uh oh!

travisghansen commented May 29, 2025

Uh oh!

k8s-ci-robot commented May 29, 2025

Uh oh!

k8s-triage-robot commented Jun 28, 2025

Uh oh!

k8s-ci-robot commented Jun 28, 2025

Uh oh!

travisghansen commented Nov 20, 2025

Uh oh!

k8s-ci-robot commented Nov 20, 2025

Uh oh!

travisghansen commented Nov 22, 2025

Uh oh!

travisghansen commented Nov 22, 2025

Uh oh!

travisghansen commented Nov 22, 2025

Uh oh!

Reviewers

Assignees

pohly Mar 1, 2022 •

edited

Loading