test-cbor fails

[rex4539]

What is the issue you have?

7/51 Testing: test-cbor
7/51 Test: test-cbor
Command: "/Users/rex/json/build/test/test-cbor" "--no-skip"
Directory: /Users/rex/json
"test-cbor" start time: May 31 16:37 EEST
Output:
----------------------------------------------------------
[doctest] doctest version is "2.3.7"
[doctest] run with "--help" for options
===============================================================================
../test/src/unit-cbor.cpp:2029:
TEST CASE:  CBOR roundtrips
  input from flynn

../test/src/unit-cbor.cpp:2269: ERROR: CHECK( vec == packed ) is NOT correct!
  values: CHECK( {?} == {?} )
  logged: filename := /Users/rex/json/build/json_test_data/json_roundtrip/roundtrip20.json
          /Users/rex/json/build/json_test_data/json_roundtrip/roundtrip20.json: output to output adapters
          /Users/rex/json/build/json_test_data/json_roundtrip/roundtrip20.json: output adapters: std::vector<uint8_t>

../test/src/unit-cbor.cpp:2269: ERROR: CHECK( vec == packed ) is NOT correct!
  values: CHECK( {?} == {?} )
  logged: filename := /Users/rex/json/build/json_test_data/json_roundtrip/roundtrip21.json
          /Users/rex/json/build/json_test_data/json_roundtrip/roundtrip21.json: output to output adapters
          /Users/rex/json/build/json_test_data/json_roundtrip/roundtrip21.json: output adapters: std::vector<uint8_t>

../test/src/unit-cbor.cpp:2269: ERROR: CHECK( vec == packed ) is NOT correct!
  values: CHECK( {?} == {?} )
  logged: filename := /Users/rex/json/build/json_test_data/nst_json_testsuite/test_parsing/y_number_0e+1.json
          /Users/rex/json/build/json_test_data/nst_json_testsuite/test_parsing/y_number_0e+1.json: output to output adapters
          /Users/rex/json/build/json_test_data/nst_json_testsuite/test_parsing/y_number_0e+1.json: output adapters: std::vector<uint8_t>

../test/src/unit-cbor.cpp:2269: ERROR: CHECK( vec == packed ) is NOT correct!
  values: CHECK( {?} == {?} )
  logged: filename := /Users/rex/json/build/json_test_data/nst_json_testsuite/test_parsing/y_number_0e1.json
          /Users/rex/json/build/json_test_data/nst_json_testsuite/test_parsing/y_number_0e1.json: output to output adapters
          /Users/rex/json/build/json_test_data/nst_json_testsuite/test_parsing/y_number_0e1.json: output adapters: std::vector<uint8_t>

../test/src/unit-cbor.cpp:2269: ERROR: CHECK( vec == packed ) is NOT correct!
  values: CHECK( {?} == {?} )
  logged: filename := /Users/rex/json/build/json_test_data/nst_json_testsuite/test_parsing/y_number_double_huge_neg_exp.json
          /Users/rex/json/build/json_test_data/nst_json_testsuite/test_parsing/y_number_double_huge_neg_exp.json: output to output adapters
          /Users/rex/json/build/json_test_data/nst_json_testsuite/test_parsing/y_number_double_huge_neg_exp.json: output adapters: std::vector<uint8_t>

../test/src/unit-cbor.cpp:2269: ERROR: CHECK( vec == packed ) is NOT correct!
  values: CHECK( {?} == {?} )
  logged: filename := /Users/rex/json/build/json_test_data/nst_json_testsuite/test_parsing/y_number_int_with_exp.json
          /Users/rex/json/build/json_test_data/nst_json_testsuite/test_parsing/y_number_int_with_exp.json: output to output adapters
          /Users/rex/json/build/json_test_data/nst_json_testsuite/test_parsing/y_number_int_with_exp.json: output adapters: std::vector<uint8_t>

../test/src/unit-cbor.cpp:2269: ERROR: CHECK( vec == packed ) is NOT correct!
  values: CHECK( {?} == {?} )
  logged: filename := /Users/rex/json/build/json_test_data/nst_json_testsuite/test_parsing/y_number_real_capital_e_pos_exp.json
          /Users/rex/json/build/json_test_data/nst_json_testsuite/test_parsing/y_number_real_capital_e_pos_exp.json: output to output adapters
          /Users/rex/json/build/json_test_data/nst_json_testsuite/test_parsing/y_number_real_capital_e_pos_exp.json: output adapters: std::vector<uint8_t>

../test/src/unit-cbor.cpp:2269: ERROR: CHECK( vec == packed ) is NOT correct!
  values: CHECK( {?} == {?} )
  logged: filename := /Users/rex/json/build/json_test_data/nst_json_testsuite/test_parsing/y_number_real_pos_exponent.json
          /Users/rex/json/build/json_test_data/nst_json_testsuite/test_parsing/y_number_real_pos_exponent.json: output to output adapters
          /Users/rex/json/build/json_test_data/nst_json_testsuite/test_parsing/y_number_real_pos_exponent.json: output adapters: std::vector<uint8_t>

../test/src/unit-cbor.cpp:2269: ERROR: CHECK( vec == packed ) is NOT correct!
  values: CHECK( {?} == {?} )
  logged: filename := /Users/rex/json/build/json_test_data/nst_json_testsuite/test_parsing/y_number_real_underflow.json
          /Users/rex/json/build/json_test_data/nst_json_testsuite/test_parsing/y_number_real_underflow.json: output to output adapters
          /Users/rex/json/build/json_test_data/nst_json_testsuite/test_parsing/y_number_real_underflow.json: output adapters: std::vector<uint8_t>

===============================================================================
[doctest] test cases:      6 |      5 passed |      1 failed |      0 skipped
[doctest] assertions: 1641795 | 1641786 passed |      9 failed |
[doctest] Status: FAILURE!
<end of output>
Test time = 102.51 sec
----------------------------------------------------------
Test Failed.
"test-cbor" end time: May 31 16:38 EEST
"test-cbor" time elapsed: 00:01:42
----------------------------------------------------------

Please describe the steps to reproduce the issue.

1. ninja test

Which compiler and operating system are you using?

• Compiler: Apple clang version 11.0.3 (clang-1103.0.32.62)
• Operating system: macOS 10.15.5 (19F94a)

Which version of the library did you use?

• latest release version 3.7.3
• other release - please state the version: ___
• the develop branch

[dota17]

@rex4539 which version of json test data did you use?

[rex4539]

Not sure what you mean. I haven't tinkered at all with anything.

Here is my build config.

% cmake -L
-- Cache values
BUILD_TESTING:BOOL=ON
CMAKE_BUILD_TYPE:STRING=Debug
CMAKE_EXECUTABLE_FORMAT:STRING=MACHO
CMAKE_INSTALL_PREFIX:PATH=/usr/local
CMAKE_OSX_ARCHITECTURES:STRING=
CMAKE_OSX_DEPLOYMENT_TARGET:STRING=
CMAKE_OSX_SYSROOT:PATH=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk
JSON_BuildTests:BOOL=ON
JSON_Coverage:BOOL=OFF
JSON_Install:BOOL=ON
JSON_MultipleHeaders:BOOL=OFF
JSON_NoExceptions:BOOL=OFF
JSON_Sanitizer:BOOL=OFF
JSON_Valgrind:BOOL=OFF
LSB_RELEASE_COMMAND:FILEPATH=LSB_RELEASE_COMMAND-NOTFOUND
SW_VERS_COMMAND:FILEPATH=/usr/bin/sw_vers
UNAME_COMMAND:FILEPATH=/usr/bin/uname
VER_COMMAND:FILEPATH=VER_COMMAND-NOTFOUND

[dota17]

well ,the test data source you use is in a separate repository, and it has updated some data 20 days ago.
So, I guess you might still use old test data.

[rex4539]

Yes, that was it. I wonder why the test data weren't updated on repo then.

I suppose the test suite doesn't check for updated test data every time it's ran.

So I really need to manually remove the folder to force it to redownload on every test run, in order to make sure I always have the latest test data?