Update Kueue and Jobset controller default limit value #502

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Open

ycchenzheng wants to merge 10 commits into AI-Hypercomputer:develop from ycchenzheng:chzheng/controller-manager-limit

ycchenzheng commented Jun 16, 2025 •

edited

Loading

Fixes / Features

Increase the memory limit of kueue-controller-manager and jobset-controller-manager to 1.2Mib per VM or 4Gi which is greater. The change was originally from b/421199006 and b/418017963. The change is implemented in utils

Testing / Documentation

Testing details.

[ y ] Tests pass
[ y ] Appropriate changes to documentation are included in the PR


          Update Kueue and Jobset controller default limit value

a875fdf

ycchenzheng requested review from Obliviour, 44past4, sharabiani, pawloch00, BluValor, gcie and RoshaniN as code owners

June 16, 2025 19:49

Author

ycchenzheng commented Jun 16, 2025


          Update cluster.py

52915c9

RoshaniN reviewed

View reviewed changes

Collaborator

RoshaniN left a comment

Thanks Zheng!

Had couple of questions.

src/xpk/utils/kubectl.py Outdated Show resolved Hide resolved

src/xpk/utils/kubectl.py Outdated Show resolved Hide resolved

ycchenzheng requested a review from RoshaniN

June 17, 2025 01:32

SujeethJinesh reviewed

View reviewed changes

src/xpk/utils/kubectl.py Outdated Show resolved Hide resolved

SujeethJinesh reviewed

View reviewed changes

src/xpk/utils/kubectl.py Outdated Show resolved Hide resolved

pawloch00 reviewed

View reviewed changes

src/xpk/utils/kubectl.py Outdated Show resolved Hide resolved

pawloch00 reviewed

View reviewed changes

src/xpk/utils/kubectl.py Outdated Show resolved Hide resolved

pawloch00 reviewed

View reviewed changes

src/xpk/utils/kubectl.py Outdated Show resolved Hide resolved

pawloch00 reviewed

View reviewed changes

src/xpk/utils/kubectl.py Outdated Show resolved Hide resolved

pawloch00 reviewed

View reviewed changes

src/xpk/utils/kubectl.py Outdated Show resolved Hide resolved

pawloch00 reviewed

View reviewed changes

src/xpk/utils/kubectl.py Outdated Show resolved Hide resolved

pawloch00 reviewed

View reviewed changes

src/xpk/utils/kubectl.py Outdated Show resolved Hide resolved

pawloch00 reviewed

View reviewed changes

src/xpk/utils/kubectl.py Outdated Show resolved Hide resolved

ycchenzheng added 2 commits

June 23, 2025 15:28


          Split into get and update manifest

734a724


          Remove dup lines

f1a26d8

SujeethJinesh reviewed

View reviewed changes

src/xpk/commands/cluster.py Outdated Show resolved Hide resolved

src/xpk/utils/kubectl.py Outdated Show resolved Hide resolved

src/xpk/utils/kubectl.py Outdated Show resolved Hide resolved

src/xpk/utils/kubectl.py Outdated Show resolved Hide resolved

src/xpk/utils/kubectl.py Outdated Show resolved Hide resolved

src/xpk/utils/kubectl.py Outdated Show resolved Hide resolved

src/xpk/utils/kubectl.py Outdated Show resolved Hide resolved


          Organize code

66e263f

ycchenzheng commented

View reviewed changes

src/xpk/commands/cluster.py Outdated Show resolved Hide resolved

ycchenzheng added 3 commits

June 25, 2025 19:18


          Redesign the feature

389a62e


          Clean up code

d15cd93


          Correct wrong description

bab8a81

ycchenzheng commented

View reviewed changes

src/xpk/core/jobset.py Show resolved Hide resolved

ycchenzheng commented

View reviewed changes

src/xpk/core/kueue.py Show resolved Hide resolved

ycchenzheng added 2 commits

June 25, 2025 22:05


          Merge branch 'develop' into chzheng/controller-manager-limit


          Remove unnecessary section of yaml

c4719a9

ycchenzheng requested review from pawloch00, wstcliyu and SujeethJinesh

June 26, 2025 02:10

wstcliyu approved these changes

View reviewed changes

wstcliyu left a comment

LGTM. Please also reach out to XPK owners for verification. @Obliviour @pawloch00

Obliviour approved these changes

View reviewed changes

Collaborator

Obliviour left a comment

Thanks for this change, looking forward to avoiding limits as we scale. Can we confirm (if not already) with GKE team that these calculated values are good?

Author

ycchenzheng commented Jul 1, 2025

@SujeethJinesh can you please take a look at this comment?

DannyLiCom commented Jul 3, 2025

@Obliviour Can you contact the GKE Team? Because we want to also create a tracking buganizer ticket for this issue/question, and then assign it to them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Obliviour Obliviour approved these changes

wstcliyu wstcliyu approved these changes

44past4 Awaiting requested review from 44past4 44past4 is a code owner

sharabiani Awaiting requested review from sharabiani sharabiani is a code owner

BluValor Awaiting requested review from BluValor BluValor is a code owner

gcie Awaiting requested review from gcie gcie is a code owner

RoshaniN Awaiting requested review from RoshaniN RoshaniN is a code owner

pawloch00 Awaiting requested review from pawloch00 pawloch00 is a code owner

SujeethJinesh Awaiting requested review from SujeethJinesh

Labels

None yet