Skip to content

Bareminimum workqueue test #263

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
May 20, 2025
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 87 additions & 0 deletions pkg/workqueue/workqueue_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
/*
* Copyright (c) 2025 NVIDIA CORPORATION. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package workqueue

import (
"context"
"sync/atomic"
"testing"
"time"

"github.com/stretchr/testify/require"
"k8s.io/apimachinery/pkg/runtime"
)

func TestEnqueue(t *testing.T) {
// Create a WorkQueue using the default rate limiter.
defaultRateLimiter := DefaultControllerRateLimiter()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which other rate limiter could we have chosen here? Why do we pick the default rate limiter? What's the advantage or disadvantage?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wq := New(defaultRateLimiter)

require.NotNil(t, wq)
require.NotNil(t, wq.queue)

// Create a context with timeout for processing.
// use DefaultTypedControllerRateLimiter Base delay: 5ms
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Millisecond)
defer cancel()

// Test EnqueueRaw
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To outsiders (like myself!), this comment does not explain what this test is about. This comment would have been the perfect opportunity to tell me. :)

So, for the future this means: a good code comment tells the part of the story that the code by itself does not tell. It shortens the time it takes to understand what code is doing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noted.

t.Run("EnqueueRaw", func(t *testing.T) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the behaviour of EnqueueRaw should be a different test suite. We want to test similar test cases as for Enqueue.

var called int32
callback := func(ctx context.Context, obj any) error {
atomic.StoreInt32(&called, 1)
return nil
}
wq.EnqueueRaw("AnyObject", callback)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@elezar sure i can add more individuals tests similar to Enqueue. This will not even go to the callback check as it will throw error on invalid obj.

wq.processNextWorkItem(ctx)

if atomic.LoadInt32(&called) != 1 {
t.Error("EnqueueRaw callback was not invoked")
}
Comment on lines +52 to +54
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this sufficient? Should we not also check that the callback was called with the expected values / object?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would have expected a reply to this good question :). What's sufficient or not for now (in this patch) is of course up to our own discretion.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My thought was to keep the test aligned with what workqueue.go does:

  • add the right item (valid object and callback func) and reject the incomplete
  • process it
  • handle retry

Looking at the workqueue.go, we only invoke the callback. I think the computeDomain controller code would be a better place to actually test the callbacks. Because most action happens there as I recall.

})
// Test Enqueue with valid and invalid runtime.Object and nil callback
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You probably notice yourself: this is another code comment that doesn't really say more than code does already.

// TODO: Implement a proper claim spec that needs to be processed
t.Run("EnqueueValid", func(t *testing.T) {
var called int32
callback := func(ctx context.Context, obj any) error {
_, ok := obj.(runtime.Object)
if !ok {
t.Errorf("Expected runtime.Object, got %T", obj)
}
atomic.StoreInt32(&called, 1)
return nil
}
validObj := &runtime.Unknown{}
wq.Enqueue(validObj, callback)
wq.processNextWorkItem(ctx)

if atomic.LoadInt32(&called) != 1 {
t.Error("Enqueue callback was not invoked")
}
Comment on lines +72 to +74
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you comment on why it's required to use atomics in the context of the tests?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I read that this is a standard go practice to use atomics to test callback functions

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be nice to have a bit deeper understanding here what type of challenge the usage of atomics really addresses in this particular context. I also don't understand that yet. We'll see.

I just had a quick web search on the topic and found the new and shiny https://go.dev/blog/synctest -- looks interesting.

})

t.Run("EnqueueInvalid", func(t *testing.T) {
callback := func(ctx context.Context, obj any) error { return nil }
wq.Enqueue("NotRuntimeObject", callback)
})

t.Run("NilCallback", func(t *testing.T) {
validObj := &runtime.Unknown{}
wq.Enqueue(validObj, nil)
wq.processNextWorkItem(ctx)
})
Comment on lines +30 to +86
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// Create a WorkQueue using the default rate limiter.
defaultRateLimiter := DefaultControllerRateLimiter()
wq := New(defaultRateLimiter)
require.NotNil(t, wq)
require.NotNil(t, wq.queue)
// Create a context with timeout for processing.
// use DefaultTypedControllerRateLimiter Base delay: 5ms
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Millisecond)
defer cancel()
// Test EnqueueRaw
t.Run("EnqueueRaw", func(t *testing.T) {
var called int32
callback := func(ctx context.Context, obj any) error {
atomic.StoreInt32(&called, 1)
return nil
}
wq.EnqueueRaw("AnyObject", callback)
wq.processNextWorkItem(ctx)
if atomic.LoadInt32(&called) != 1 {
t.Error("EnqueueRaw callback was not invoked")
}
})
// Test Enqueue with valid and invalid runtime.Object and nil callback
// TODO: Implement a proper claim spec that needs to be processed
t.Run("EnqueueValid", func(t *testing.T) {
var called int32
callback := func(ctx context.Context, obj any) error {
_, ok := obj.(runtime.Object)
if !ok {
t.Errorf("Expected runtime.Object, got %T", obj)
}
atomic.StoreInt32(&called, 1)
return nil
}
validObj := &runtime.Unknown{}
wq.Enqueue(validObj, callback)
wq.processNextWorkItem(ctx)
if atomic.LoadInt32(&called) != 1 {
t.Error("Enqueue callback was not invoked")
}
})
t.Run("EnqueueInvalid", func(t *testing.T) {
callback := func(ctx context.Context, obj any) error { return nil }
wq.Enqueue("NotRuntimeObject", callback)
})
t.Run("NilCallback", func(t *testing.T) {
validObj := &runtime.Unknown{}
wq.Enqueue(validObj, nil)
wq.processNextWorkItem(ctx)
})
wq := New(DefaultControllerRateLimiter())
require.NotNil(t, wq)
require.NotNil(t, wq.queue)
ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond)
defer cancel()
tests := []struct {
name string
obj any
callback func(context.Context, any) error
validate func(*testing.T, *int32)
processNextWorkItem bool
}{
{
name: "EnqueueRaw",
obj: "AnyObject",
callback: func(ctx context.Context, obj any) error {
atomic.StoreInt32(obj.(*int32), 1)
return nil
},
validate: func(t *testing.T, called *int32) {
if atomic.LoadInt32(called) != 1 {
t.Error("EnqueueRaw callback was not invoked")
}
},
processNextWorkItem: true,
},
{
name: "EnqueueValid",
obj: &runtime.Unknown{},
callback: func(ctx context.Context, obj any) error {
if _, ok := obj.(runtime.Object); !ok {
t.Errorf("Expected runtime.Object, got %T", obj)
}
atomic.StoreInt32(new(int32), 1)
return nil
},
validate: func(t *testing.T, called *int32) {
// No validation needed for runtime.Object type
},
processNextWorkItem: true,
},
{
name: "EnqueueInvalid",
obj: "NotRuntimeObject",
callback: func(ctx context.Context, obj any) error { return nil },
validate: nil,
processNextWorkItem: false,
},
{
name: "NilCallback",
obj: &runtime.Unknown{},
callback: nil,
validate: nil,
processNextWorkItem: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
var called int32
if tt.name == "EnqueueRaw" {
wq.EnqueueRaw(&called, tt.callback)
} else {
wq.Enqueue(tt.obj, tt.callback)
}
if tt.processNextWorkItem {
wq.processNextWorkItem(ctx)
if tt.validate != nil {
tt.validate(t, &called)
}
}
})
}

I have tested this suggestion it works

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO, it may be a bit of an overkill to add the extra validate func and bool just to make the test compact. If its not strongly desired, I would like to stick to the current format.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ArangoGutierrez how do we move forward with this? :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ArangoGutierrez how do we move forward with this? :)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a not-so-bad answer at this point is -- let's just move.

}