Fix input_needed_for_output to not return one less than needed, causing draining down the line #811

padenot · 2025-01-23T15:40:57Z

The calculation was incorrect. This meant that in some circumstances,
e.g. playing a 44.1Hz stream on a 96kHz interface, with a block of 441
frames on Windows, rounding was eventually returning 440 instead of 441
in the callback, leading to the stream entering the draining phase.

padenot · 2025-01-23T15:46:26Z

This fixes BMO#1940319.

test/test_resampler.cpp

karlt

I see now that github interprets review of a commit as review of the entire pull request.

Thank you for testing thoroughly.
I expect that this function should be able to be simple without special cases. The special cases make me suspect that something else may not be quite right.

src/cubeb_resampler_internal.h

karlt

I spent several hours trying to prove that a certain calculation would produce enough input for an output, but I have not yet come up with something concrete.

The resampler advance can be understood from the code in resampler_basic_zero() (or any of the other resampler_basic_funcs).

The resampler can be considered to count at num_rate * den_rate quantums per
second, where den_rate = out_rate / GCF(in_rate, out_rate) and num_rate = in_rate / GCF(in_rate, out_rate).

samp_frac_num counts in these quantums.
Each last_sample unit represents den_rate quantums.
Adding the two together gives a position.

Each resampler iteration advances by num_rate quantums (1/den_rate seconds), and produces one output frame.

Initially the first input and output frames correspond to 0 quantums.

Each speex_resample() call leaves the resampler at a position exactly aligned with an output frame, so output_frame_count output frames requires an advance of output_frame_count * num_rate quantums.

Input frames intervals are den_rate quantums, but the first input frame is aligned differently on different speex_resample() calls, and there is an additional complicating factor that the speex resampler will sometimes consume input frames that it doesn't actually use.

last_sample values correspond to input frame positions, but last_sample can get ahead of the number of input frames consumed. In the last iteration of the resampler_basic_func, last_sample can advance beyond *in_len. The input frames stepped over are not needed yet. Input frames are consumed up to last_sample, even the frames that weren't needed.

The consuming of some unused input frames aligns better with the expected ratio of input to output frames given the alignment at 0 quantums. When last_sample advances beyond *in_len, if those extra frames are not yet available, they are pulled on the next speex_resample() call, pulling more frames than expected.

src/cubeb_resampler_internal.h

padenot · 2025-01-28T17:44:32Z

I'll probably add some test that checks that the output is sensible, iirc we have an utility for that somewhere (feed a sine wave at a certain freq, check that we get that at the output).

karlt

Thanks!

src/cubeb_resampler_internal.h

test/test_resampler.cpp

…ng draining down the line The calculation was incorrect. This meant that in some circumstances, e.g. playing a 44.1Hz stream on a 96kHz interface, with a block of 441 frames on Windows, rounding was eventually returning 440 instead of 441 in the callback, leading to the stream entering the draining phase.

…ck sizes and sample-rates This might be a bit much, runtime is 25s debug ASAN, 6s opt, but should be thorough.

karlt

The large tolerance implies that something is not right either in the continuity test or in what it is testing.

Would you like to drop that commit from the PR and deal with that separately, to get the draining fix landed?
Or do you think the continuity test is important to check that the drain fix is not breaking continuity?

test/test_resampler.cpp

karlt · 2025-02-10T09:41:50Z

test/test_resampler.cpp

+          // `data`. This isn't stricly necessary but helps having cleaner
+          // dumps for manual inspection in e.g. Audacity. In all our test cases


The MSE would be much higher without this, because the initial silence is very different from a sine wave, so I assume this stripping is necessary.

karlt · 2025-02-10T09:43:12Z

test/test_resampler.cpp

+// Finds the offset of the start of an input_freq sine wave sampled at
+// target_rate in data. Remove the leading silence from data.
+size_t
+find_sine_start(std::vector<float> & data, float input_freq, float target_rate)


When I read the caller, I assumed this was a pure function.
How about skip_to_sine_start()?

karlt · 2025-02-10T10:38:00Z

test/test_resampler.cpp

+        // 1.5 is a good higher bound found empirically. Any mistake, e.g.
+        // discontinuity make the MSE shoot up dramatically.
+        ASSERT_LT(mse, 1.5);


1.5 seems a high tolerance for a mean squared error between samples in the range [-0.8, 0.8]. Comparing 0.8 * sin() with zero would have a mean squared error of about 0.32, if my maths is correct. I would expect the MSE for the resampler output to be a small fraction of 1 if everything is behaving as intended.

A large error in a single sample is greatly attenuated by a mean square error.
A maximum error might be better for our purposes, and easier to reason about a tolerance.

karlt · 2025-02-10T10:40:48Z

test/test_resampler.cpp

+  for (size_t i = 0; i < len; ++i) {
+    float c = std::cos(phase_incr * static_cast<float>(i));
+    float s = std::sin(phase_incr * static_cast<float>(i));
+    sum_cos += signal[i] * c;
+    sum_sin += signal[i] * s;
+  }
+
+  float amplitude = 2.0f * std::sqrt(sum_cos * sum_cos + sum_sin * sum_sin) /
+                    static_cast<float>(len);
+  float phi = std::atan2(sum_sin, sum_cos);


I think I may have discarded my copy of Numerical Methods for Engineers.
This is a fancy simplification of what was presented in the other references.
I understand dropping of the column of 1s from D_0 because we expect zero DC offset.

(D_0^T D_0)^-1 has been replaced with 2/n * I.
Do you have a reference for that?
I suspect that would not hold exactly if the signal duration is not an integer multiple of the period (or perhaps half period?). The skipping of leading samples means that the duration is not an integer multiple, so this formulation would be an approximation for duration >> period.
I suspect this may be related to "When frequencies are chosen precisely as recommended for coherent sampling, with an exact integer number of cycles in a record, a DFT will give the same results as a three-parameter sine wave fit."

If such an approximation is good enough, then some explanation would be helpful.
Doing the full calculation of D_0^T D_0 would not be a lot more code.

karlt · 2025-02-10T11:07:31Z

test/test_resampler.cpp

+
+        cubeb_resampler * resampler = cubeb_resampler_create(
+            nullptr, nullptr, &out_params, source_rate, data_cb, &state,
+            CUBEB_RESAMPLER_QUALITY_DEFAULT, CUBEB_RESAMPLER_RECLOCK_NONE);


Did you intentionally change this back to _DEFAULT in "Test improvements: check continuity"?

karlt

I'll hold off on looking at the threading until the test seems to be passing.

karlt · 2025-02-13T05:58:46Z

CMakeLists.txt

@@ -51,6 +51,7 @@ if(BUILD_TESTS)
      message(FATAL_ERROR "Could not find googletest: run\n\tgit submodule update --init --recursive\nin base git checkout")
    endif()
    add_definitions(-DGTEST_HAS_TR1_TUPLE=0 -DGTEST_HAS_RTTI=0)
+    add_definitions(-DGTEST_ENABLED=1)


With 8ea49e1, I have

~/scratch/cubeb/git% mkdir build ~/scratch/cubeb/git% cd build /var/karl/cubeb/git/build % cmake ../ -DCMAKE_BUILD_TYPE=Debug -B . -- The C compiler identification is GNU 13.3.1 -- The CXX compiler identification is GNU 13.3.1 -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: /bin/cc - skipped -- Detecting C compile features -- Detecting C compile features - done -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: /bin/c++ - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY -- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY - Success -- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY -- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY - Success -- Performing Test COMPILER_HAS_DEPRECATED_ATTR -- Performing Test COMPILER_HAS_DEPRECATED_ATTR - Success -- Found PkgConfig: /bin/pkg-config (found version "2.3.0") -- Checking for module 'speexdsp' -- Package 'speexdsp' not found -- Found Threads: TRUE -- Looking for include file pulse/pulseaudio.h -- Looking for include file pulse/pulseaudio.h - found -- Looking for include file alsa/asoundlib.h -- Looking for include file alsa/asoundlib.h - found -- Looking for include file jack/jack.h -- Looking for include file jack/jack.h - not found -- Looking for include file sndio.h -- Looking for include file sndio.h - not found -- Looking for include file aaudio/AAudio.h -- Looking for include file aaudio/AAudio.h - not found -- Looking for include file AudioUnit/AudioUnit.h -- Looking for include file AudioUnit/AudioUnit.h - not found -- Looking for include file audioclient.h -- Looking for include file audioclient.h - not found -- Looking for include files windows.h, mmsystem.h -- Looking for include files windows.h, mmsystem.h - not found -- Looking for include file SLES/OpenSLES.h -- Looking for include file SLES/OpenSLES.h - not found -- Looking for include file sys/soundcard.h -- Looking for include file sys/soundcard.h - found -- Looking for include file android/log.h -- Looking for include file android/log.h - not found -- Looking for include file sys/audioio.h -- Looking for include file sys/audioio.h - not found -- Looking for include file kai.h -- Looking for include file kai.h - not found -- Found Doxygen: /bin/doxygen (found version "1.12.0") found components: doxygen missing components: dot -- Configuring done (1.2s) -- Generating done (0.0s) -- Build files have been written to: /home/karl/scratch/cubeb/git/build ~/scratch/cubeb/git/build% make [...] [ 52%] Building CXX object CMakeFiles/test_resampler.dir/test/test_resampler.cpp.o /home/karl/scratch/cubeb/git/test/test_resampler.cpp: In member function 'virtual void cubeb_resampler_typical_uses_Test::TestBody()': /home/karl/scratch/cubeb/git/test/test_resampler.cpp:1424:11: error: 'cubeb_resampler_stats' was not declared in this scope; did you mean 'cubeb_resampler_speex'? 1424 | cubeb_resampler_stats stats = cubeb_resampler_stats_get(resampler); | ^~~~~~~~~~~~~~~~~~~~~ | cubeb_resampler_speex /home/karl/scratch/cubeb/git/test/test_resampler.cpp:1433:35: error: 'stats' was not declared in this scope; did you mean 'statx'? 1433 | in_in_max.set_new_value(stats.input_input_buffer_size); | ^~~~~ | statx make[2]: *** [CMakeFiles/test_resampler.dir/build.make:76: CMakeFiles/test_resampler.dir/test/test_resampler.cpp.o] Error 1 make[1]: *** [CMakeFiles/Makefile2:363: CMakeFiles/test_resampler.dir/all] Error 2 make: *** [Makefile:146: all] Error 2 ~/scratch/cubeb/git/build2% grep GTEST_ENABLED *(.) ~/scratch/cubeb/git/build1%

The subsequently posted changes wouldn't change that.

karlt · 2025-02-13T06:06:59Z

test/test_resampler.cpp

+
+  float amplitude = 2.0f * std::sqrt(sum_cos * sum_cos + sum_sin * sum_sin) /
+                    static_cast<float>(len);
+  float phi = std::atan2(sum_sin, sum_cos);


phi = std::atan2(sum_sin, sum_cos) is for std::cos(phase_incr * i - phi).
I suspect phi = std::atan2(sum_cos, sum_sin) would be appropriate for std::sin(phase_incr * i + phi). That improves the MSE in some cases so that it is less than 0.0001.

In other cases the MSE is still close to 0.32 because the fitted amplitude is small. The signal is not close to zero, which implies that the fitting of the sine is failing. I don't know why that is. Perhaps it might plausibly be due to accumulation of rounding causing phase variation in the source signal, but i have not observed clear evidence for or against that.

karlt approved these changes Jan 27, 2025

View reviewed changes

test/test_resampler.cpp Outdated Show resolved Hide resolved

test/test_resampler.cpp Outdated Show resolved Hide resolved

karlt approved these changes Jan 27, 2025

View reviewed changes

karlt suggested changes Jan 27, 2025

View reviewed changes

src/cubeb_resampler_internal.h Show resolved Hide resolved

src/cubeb_resampler_internal.h Show resolved Hide resolved

src/cubeb_resampler_internal.h Outdated Show resolved Hide resolved

src/cubeb_resampler_internal.h Outdated Show resolved Hide resolved

karlt reviewed Jan 28, 2025

View reviewed changes

src/cubeb_resampler_internal.h Outdated Show resolved Hide resolved

src/cubeb_resampler_internal.h Outdated Show resolved Hide resolved

padenot force-pushed the resampler-draining branch from 8f7e192 to 1b52d0f Compare January 28, 2025 17:40

padenot force-pushed the resampler-draining branch from 1b52d0f to 3b1bcae Compare January 28, 2025 17:51

karlt approved these changes Jan 29, 2025

View reviewed changes

src/cubeb_resampler_internal.h Outdated Show resolved Hide resolved

src/cubeb_resampler_internal.h Outdated Show resolved Hide resolved

src/cubeb_resampler_internal.h Outdated Show resolved Hide resolved

test/test_resampler.cpp Outdated Show resolved Hide resolved

padenot added 7 commits February 3, 2025 16:29

Add carriage return when logging in tests

c8b2c9f

Add test-only cubeb_resampler introspection API

2cff1b8

Add systematic integration test for cubeb_resampler, with typical blo…

6050c90

…ck sizes and sample-rates This might be a bit much, runtime is 25s debug ASAN, 6s opt, but should be thorough.

Run clang-format

b01c41e

Code fixes

bbd23de

Fix leak in cubeb_audio_dump.cpp

00db3a7

padenot force-pushed the resampler-draining branch 2 times, most recently from 5ca5a4e to 56d90bd Compare February 3, 2025 15:47

padenot added 2 commits February 4, 2025 14:50

Test improvements: check continuity

2be161e

Add .cache to gitignore, created by clangd

8ea49e1

padenot force-pushed the resampler-draining branch from 56d90bd to 8ea49e1 Compare February 4, 2025 13:50

karlt suggested changes Feb 10, 2025

View reviewed changes

padenot added 2 commits February 12, 2025 12:47

constexpr dump

d1ab461

Parallelize test

5f5f266

padenot force-pushed the resampler-draining branch from 42be36d to 5f5f266 Compare February 12, 2025 11:51

karlt suggested changes Feb 13, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix input_needed_for_output to not return one less than needed, causing draining down the line #811

Fix input_needed_for_output to not return one less than needed, causing draining down the line #811

padenot commented Jan 23, 2025

padenot commented Jan 23, 2025

karlt left a comment •

edited

Loading

karlt left a comment •

edited

Loading

padenot commented Jan 28, 2025

karlt left a comment

karlt left a comment •

edited

Loading

karlt Feb 10, 2025

karlt Feb 10, 2025

karlt Feb 10, 2025

karlt Feb 10, 2025

karlt Feb 10, 2025 •

edited

Loading

karlt Feb 10, 2025

karlt left a comment

karlt Feb 13, 2025

karlt Feb 13, 2025

		// `data`. This isn't stricly necessary but helps having cleaner
		// dumps for manual inspection in e.g. Audacity. In all our test cases

Fix input_needed_for_output to not return one less than needed, causing draining down the line #811

Are you sure you want to change the base?

Fix input_needed_for_output to not return one less than needed, causing draining down the line #811

Conversation

padenot commented Jan 23, 2025

padenot commented Jan 23, 2025

karlt left a comment • edited Loading

Choose a reason for hiding this comment

karlt left a comment • edited Loading

Choose a reason for hiding this comment

padenot commented Jan 28, 2025

karlt left a comment

Choose a reason for hiding this comment

karlt left a comment • edited Loading

Choose a reason for hiding this comment

karlt Feb 10, 2025

Choose a reason for hiding this comment

karlt Feb 10, 2025

Choose a reason for hiding this comment

karlt Feb 10, 2025

Choose a reason for hiding this comment

karlt Feb 10, 2025

Choose a reason for hiding this comment

karlt Feb 10, 2025 • edited Loading

Choose a reason for hiding this comment

karlt Feb 10, 2025

Choose a reason for hiding this comment

karlt left a comment

Choose a reason for hiding this comment

karlt Feb 13, 2025

Choose a reason for hiding this comment

karlt Feb 13, 2025

Choose a reason for hiding this comment

karlt left a comment •

edited

Loading

karlt left a comment •

edited

Loading

karlt left a comment •

edited

Loading

karlt Feb 10, 2025 •

edited

Loading