Flaky e2e TestModelServingParitionScaleDown

[verma-garv]

https://github.com/volcano-sh/kthena/actions/runs/28990791465/job/86029868231?pr=1326#step:7:9972

    model_serving_test.go:1501: Scaling down from 5 to 3 replicas while partition=3
    model_serving_test.go:1505: Waiting for ModelServing to be ready...
    model_serving_test.go:1527: 
        	Error Trace:	/home/runner/work/kthena/kthena/test/e2e/controller-manager/model_serving_test.go:1527
        	Error:      	Not equal: 
        	            	expected: 3
        	            	actual  : 4
        	Test:       	TestModelServingPartitionScaleDown
--- FAIL: TestModelServingPartitionScaleDown (12.04s)

[verma-garv]

/assign