Skip to content

Reduce timeout for unit tests #1907

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Reduce timeout for unit tests #1907

wants to merge 2 commits into from

Conversation

jefchien
Copy link
Contributor

@jefchien jefchien commented Oct 16, 2025 •
edited
Loading

Description of the issue

The TestConcurrentLinePoolAccess test is flaky and can get into a situation where the test will never finish.

panic: test timed out after 15m0s
	running tests:
		TestConcurrentLinePoolAccess (14m57s)

goroutine 20 [running]:
testing.(*M).startAlarm.func1()
	C:/Users/runneradmin/go/pkg/mod/golang.org/[email protected]/src/testing/testing.go:2484 +0x394
created by time.goFunc
	C:/Users/runneradmin/go/pkg/mod/golang.org/[email protected]/src/time/sleep.go:215 +0x2d

goroutine 1 [chan receive, 14 minutes]:
testing.(*T).Run(0xc000003880, {0x2d8701?, 0x7ffaa5db3fb0?}, 0x2e95d0)
	C:/Users/runneradmin/go/pkg/mod/golang.org/[email protected]/src/testing/testing.go:1859 +0x414
testing.runTests.func1(0xc000003880)
	C:/Users/runneradmin/go/pkg/mod/golang.org/[email protected]/src/testing/testing.go:2279 +0x37
testing.tRunner(0xc000003880, 0xc000141c70)
	C:/Users/runneradmin/go/pkg/mod/golang.org/[email protected]/src/testing/testing.go:1792 +0xcb
testing.runTests(0xc0000080a8, {0x5151c0, 0xc, 0xc}, {0x51d000?, 0x7?, 0x51be80?})
	C:/Users/runneradmin/go/pkg/mod/golang.org/[email protected]/src/testing/testing.go:2277 +0x4b4
testing.(*M).Run(0xc000062960)
	C:/Users/runneradmin/go/pkg/mod/golang.org/[email protected]/src/testing/testing.go:2142 +0x64a
main.main()
	_testmain.go:77 +0x9b

goroutine 38 [sync.WaitGroup.Wait, 14 minutes]:
sync.runtime_SemacquireWaitGroup(0xc000024c00?)
	C:/Users/runneradmin/go/pkg/mod/golang.org/[email protected]/src/runtime/sema.go:110 +0x25
sync.(*WaitGroup).Wait(0x3358f8?)
	C:/Users/runneradmin/go/pkg/mod/golang.org/[email protected]/src/sync/waitgroup.go:118 +0x48
github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail.TestConcurrentLinePoolAccess(0xc000190380)
	D:/a/amazon-cloudwatch-agent/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/tail_test.go:404 +0x392
testing.tRunner(0xc000190380, 0x2e95d0)
	C:/Users/runneradmin/go/pkg/mod/golang.org/[email protected]/src/testing/testing.go:1792 +0xcb
created by testing.(*T).Run in goroutine 1
	C:/Users/runneradmin/go/pkg/mod/golang.org/[email protected]/src/testing/testing.go:1851 +0x3f6

goroutine 40 [chan receive, 14 minutes]:
github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/watch.watch(0xc0001800c0)
	D:/a/amazon-cloudwatch-agent/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/watch/inotify_tracker.go:80 +0xc5
github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/watch.Watch(...)
	D:/a/amazon-cloudwatch-agent/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/watch/inotify_tracker.go:60
github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/watch.(*InotifyFileWatcher).ChangeEvents(0xc000180090, 0xc0001181d0, 0x0)
	D:/a/amazon-cloudwatch-agent/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/watch/inotify.go:67 +0xbb
github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail.(*Tail).watchChanges(0xc000118120)
	D:/a/amazon-cloudwatch-agent/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/tail.go:503 +0x9b
github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail.(*Tail).tailFileSync(0xc000118120)
	D:/a/amazon-cloudwatch-agent/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/tail.go:401 +0x2f1
created by github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail.TailFile in goroutine 38
	D:/a/amazon-cloudwatch-agent/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/tail.go:137 +0x52b

goroutine 39 [select, 2 minutes]:
github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail.(*Tail).exitOnDeletion(0xc000118120)
	D:/a/amazon-cloudwatch-agent/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/tail.go:693 +0x125
created by github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail.TailFile in goroutine 38
	D:/a/amazon-cloudwatch-agent/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/tail.go:134 +0x4e5

goroutine 10 [chan receive, 14 minutes]:
gopkg.in/fsnotify%2ev1.(*Watcher).Add(0xc00005eec0, {0xc0001e4040?, 0xc0001e4040?})
	C:/Users/runneradmin/go/pkg/mod/gopkg.in/[email protected]/windows.go:81 +0xf3
github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/watch.(*InotifyTracker).addWatch(0xc00005ee80, 0xc0001800c0)
	D:/a/amazon-cloudwatch-agent/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/watch/inotify_tracker.go:152 +0x2ad
github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/watch.(*InotifyTracker).run(0xc00005ee80)
	D:/a/amazon-cloudwatch-agent/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/watch/inotify_tracker.go:224 +0x497
created by github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/watch.init.func1 in goroutine 9
	D:/a/amazon-cloudwatch-agent/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/watch/inotify_tracker.go:52 +0x16e

goroutine 11 [chan send, 14 minutes, locked to thread]:
gopkg.in/fsnotify%2ev1.(*Watcher).startRead(0xc00005eec0, 0xc0000b4008)
	C:/Users/runneradmin/go/pkg/mod/gopkg.in/[email protected]/windows.go:334 +0xd7
gopkg.in/fsnotify%2ev1.(*Watcher).readEvents(0xc00005eec0)
	C:/Users/runneradmin/go/pkg/mod/gopkg.in/[email protected]/windows.go:511 +0xd88
created by gopkg.in/fsnotify%2ev1.NewWatcher in goroutine 10
	C:/Users/runneradmin/go/pkg/mod/gopkg.in/[email protected]/windows.go:46 +0x1b8

goroutine 42 [chan receive, 14 minutes]:
github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail.TestConcurrentLinePoolAccess.func2()
	D:/a/amazon-cloudwatch-agent/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/tail_test.go:397 +0xb9
created by github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail.TestConcurrentLinePoolAccess in goroutine 38
	D:/a/amazon-cloudwatch-agent/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/tail_test.go:395 +0x2d0

goroutine 43 [chan receive, 14 minutes]:
github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail.TestConcurrentLinePoolAccess.func2()
	D:/a/amazon-cloudwatch-agent/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/tail_test.go:397 +0xb9
created by github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail.TestConcurrentLinePoolAccess in goroutine 38
	D:/a/amazon-cloudwatch-agent/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/tail_test.go:395 +0x2d0

goroutine 44 [chan receive, 14 minutes]:
github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail.TestConcurrentLinePoolAccess.func2()
	D:/a/amazon-cloudwatch-agent/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/tail_test.go:397 +0xb9
created by github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail.TestConcurrentLinePoolAccess in goroutine 38
	D:/a/amazon-cloudwatch-agent/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/tail_test.go:395 +0x2d0

goroutine 45 [chan receive, 14 minutes]:
github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail.TestConcurrentLinePoolAccess.func2()
	D:/a/amazon-cloudwatch-agent/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/tail_test.go:397 +0xb9
created by github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail.TestConcurrentLinePoolAccess in goroutine 38
	D:/a/amazon-cloudwatch-agent/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/tail_test.go:395 +0x2d0

goroutine 46 [chan receive, 14 minutes]:
github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail.TestConcurrentLinePoolAccess.func2()
	D:/a/amazon-cloudwatch-agent/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/tail_test.go:397 +0xb9
created by github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail.TestConcurrentLinePoolAccess in goroutine 38
	D:/a/amazon-cloudwatch-agent/amazon-cloudwatch-agent/plugins/inputs/logfile/tail/tail_test.go:395 +0x2d0
FAIL	github.com/aws/amazon-cloudwatch-agent/plugins/inputs/logfile/tail	900.041s

https://github.com/aws/amazon-cloudwatch-agent/actions/runs/18573122156/job/52952299604?pr=1903

This makes it so that when the unit tests are run, they use the full timeout set (15 minutes). The unit tests take an average of <5 minutes.

Description of changes

Reducing the unit test run timeout to 10 minutes. Fixed TestConcurrentLinePoolAccess by ensuring that the consumer channel is always closed even if the producer times out.

License

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Tests

Ran unit tests.

Requirements

Before commiting your code, please do the following steps.

  1. Run make fmt and make fmt-sh
  2. Run make lint

Integration Tests

To run integration tests against this PR, add the ready for testing label.

@jefchien jefchien requested a review from a team as a code owner October 16, 2025 21:39
@jefchien jefchien force-pushed the fix-flaky-logfile-test branch 2 times, most recently from 58d31cc to b7686a5 Compare October 16, 2025 22:16
@jefchien jefchien added the ready for testing Indicates this PR is ready for integration tests to run label Oct 16, 2025
@jefchien jefchien force-pushed the fix-flaky-logfile-test branch 3 times, most recently from c7e7b3a to 50fa77d Compare October 20, 2025 22:29
@jefchien jefchien force-pushed the fix-flaky-logfile-test branch from 50fa77d to 7db0020 Compare October 21, 2025 16:09
@jefchien jefchien force-pushed the fix-flaky-logfile-test branch from 7db0020 to 37ca456 Compare October 21, 2025 17:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

ready for testing Indicates this PR is ready for integration tests to run

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@jefchien