Many tests fail: ERROR: CHECK( std::string_view{line_actual} == std::string_view{line_expected} ) is NOT correct!

[yurivict]

/usr/ports/devel/snitch/work-default/snitch-1.3.2/tests/approval_tests/reporter_teamcity.cpp:59: ERROR: CHECK( std::string_view{line_actual} == std::string_view{line_expected} ) is NOT correct!
  values: CHECK( ##teamCity[blockClosed name='section 2'] == ##teamCity[testFinished name='test unexpected throw in section fail' duration='*'] )
  logged: checking reporter_teamcity_full
          line_number := 132

/usr/ports/devel/snitch/work-default/snitch-1.3.2/tests/approval_tests/reporter_teamcity.cpp:59: ERROR: CHECK( std::string_view{line_actual} == std::string_view{line_expected} ) is NOT correct!
  values: CHECK( ##teamCity[blockClosed name='section 1'] == ##teamCity[testStarted name='test unexpected throw in check fail'] )
  logged: checking reporter_teamcity_full
          line_number := 133

/usr/ports/devel/snitch/work-default/snitch-1.3.2/tests/approval_tests/reporter_teamcity.cpp:59: ERROR: CHECK( std::string_view{line_actual} == std::string_view{line_expected} ) is NOT correct!
  values: CHECK( ##teamCity[testFinished name='test unexpected throw in section fail' duration='*'] == ##teamCity[testFailed name='test unexpected throw in check fail' message='*testing_reporters.cpp:*|n  unexpected std::exception caught; message: unexpected error'] )
  logged: checking reporter_teamcity_full
          line_number := 134

/usr/ports/devel/snitch/work-default/snitch-1.3.2/tests/approval_tests/reporter_teamcity.cpp:59: ERROR: CHECK( std::string_view{line_actual} == std::string_view{line_expected} ) is NOT correct!
  values: CHECK( ##teamCity[testStarted name='test unexpected throw in check fail'] == ##teamCity[testFinished name='test unexpected throw in check fail' duration='*'] )
  logged: checking reporter_teamcity_full
          line_number := 135

/usr/ports/devel/snitch/work-default/snitch-1.3.2/tests/approval_tests/reporter_teamcity.cpp:59: ERROR: CHECK( std::string_view{line_actual} == std::string_view{line_expected} ) is NOT correct!
  values: CHECK( ##teamCity[testFailed name='test unexpected throw in check fail' message='/usr/ports/devel/snitch/work-default/snitch-1.3.2/tests/testing_reporters.cpp:132|n  unexpected std::exception caught; message: unexpected error'] == ##teamCity[testStarted name='test unexpected throw in check & section fail'] )
  logged: checking reporter_teamcity_full
          line_number := 136

/usr/ports/devel/snitch/work-default/snitch-1.3.2/tests/approval_tests/reporter_teamcity.cpp:59: ERROR: CHECK( std::string_view{line_actual} == std::string_view{line_expected} ) is NOT correct!
  values: CHECK( ##teamCity[testFinished name='test unexpected throw in check fail' duration='*'] == ##teamCity[blockOpened name='section 1' description=''] )
  logged: checking reporter_teamcity_full
          line_number := 137

/usr/ports/devel/snitch/work-default/snitch-1.3.2/tests/approval_tests/reporter_teamcity.cpp:59: ERROR: CHECK( std::string_view{line_actual} == std::string_view{line_expected} ) is NOT correct!
  values: CHECK( ##teamCity[testStarted name='test unexpected throw in check & section fail'] == ##teamCity[testFailed name='test unexpected throw in check & section fail' message='*testing_reporters.cpp:*|n  unexpected std::exception caught; message: unexpected error'] )
  logged: checking reporter_teamcity_full
          line_number := 138

/usr/ports/devel/snitch/work-default/snitch-1.3.2/tests/approval_tests/reporter_teamcity.cpp:59: ERROR: CHECK( std::string_view{line_actual} == std::string_view{line_expected} ) is NOT correct!
  values: CHECK( ##teamCity[blockOpened name='section 1' description=''] == ##teamCity[blockClosed name='section 1'] )
  logged: checking reporter_teamcity_full
          line_number := 139

/usr/ports/devel/snitch/work-default/snitch-1.3.2/tests/approval_tests/reporter_teamcity.cpp:59: ERROR: CHECK( std::string_view{line_actual} == std::string_view{line_expected} ) is NOT correct!
  values: CHECK( ##teamCity[testFailed name='test unexpected throw in check & section fail' message='/usr/ports/devel/snitch/work-default/snitch-1.3.2/tests/testing_reporters.cpp:136|n  unexpected std::exception caught; message: unexpected error'] == ##teamCity[testFinished name='test unexpected throw in check & section fail' duration='*'] )
  logged: checking reporter_teamcity_full
          line_number := 140

/usr/ports/devel/snitch/work-default/snitch-1.3.2/tests/approval_tests/reporter_teamcity.cpp:59: ERROR: CHECK( std::string_view{line_actual} == std::string_view{line_expected} ) is NOT correct!
  values: CHECK( ##teamCity[blockClosed name='section 1'] == ##teamCity[testStarted name='test unexpected throw in check & section mayfail'] )
  logged: checking reporter_teamcity_full
          line_number := 141

/usr/ports/devel/snitch/work-default/snitch-1.3.2/tests/runtime_tests/section.cpp:24:
TEST CASE:  section
  nested sections abort early
          
/usr/ports/devel/snitch/work-default/snitch-1.3.2/tests/runtime_tests/section.cpp:118: FATAL ERROR: REQUIRE( framework.get_num_failures() == 1u ) is NOT correct!
  values: REQUIRE( 2 == 1 )

Version: 1.3.2
clang-19
FreeBSD 14.2

[cschreib]

Thank you for reporting this. I have just tried building with clang-19 locally (Ubuntu 22.04) and all tests passed. I don't have a FreeBSD install available to check if this is specific to this OS, unfortunately. I tried setting up a VM, but I don't do this regularly, and encountered issues with secure boot and couldn't get it to work.

Are you able to share more information, for example, how exactly did you build snitch (CMake? meson?) and with what compile-time options?