failure of replication tests in the test suite

[ccacciari]

Running the test suite in iRODS 4.2.3, the last 12 tests about replication fail:

test_70_b2safe_local_for_one_file_not_registered_non_recursive (testB2SafeCmd.irodsb2safetest.IrodsB2safeIntegrationTests)
Test : replicate a single file locally using b2safe (not registered, non recursive) ... FAIL
test_70_b2safe_local_for_one_file_not_registered_recursive (testB2SafeCmd.irodsb2safetest.IrodsB2safeIntegrationTests)
Test : replicate a single file locally using b2safe (not registered,     recursive) ... FAIL
test_70_b2safe_local_for_one_file_registered_non_recursive_01 (testB2SafeCmd.irodsb2safetest.IrodsB2safeIntegrationTests)
Test : replicate a single file locally using b2safe (    registered, non recursive, ror=pid, fio=pid, fixed_content=false) ... FAIL
test_70_b2safe_local_for_one_file_registered_non_recursive_02 (testB2SafeCmd.irodsb2safetest.IrodsB2safeIntegrationTests)
Test : replicate a single file locally using b2safe (    registered, non recursive, ror=841/test, fio=None, fixed_content=false) ... FAIL
test_70_b2safe_local_for_one_file_registered_non_recursive_03 (testB2SafeCmd.irodsb2safetest.IrodsB2safeIntegrationTests)
Test : replicate a single file locally using b2safe (    registered, non recursive, ror=None, fio=None, fixed_content=false) ... FAIL
test_70_b2safe_local_for_one_file_registered_recursive (testB2SafeCmd.irodsb2safetest.IrodsB2safeIntegrationTests)
Test : replicate a single file locally using b2safe (    registered,     recursive, ror=pid, fio=pid, fixed_content=false) ... FAIL
test_75_b2safe_local_for_one_directory_not_registered_recursive (testB2SafeCmd.irodsb2safetest.IrodsB2safeIntegrationTests)
Test : replicate a single directory locally using b2safe (not registered,    recursive) ... FAIL
test_75_b2safe_local_for_one_directory_registered_recursive (testB2SafeCmd.irodsb2safetest.IrodsB2safeIntegrationTests)
Test : replicate a single directory locally using b2safe (    registered,    recursive, ror=pid, fio=pid, fixed_content=false) ... FAIL
test_80_b2safe_local_for_multiple_directories_not_registered_recursive (testB2SafeCmd.irodsb2safetest.IrodsB2safeIntegrationTests)
Test : replicate multiple directories locally using b2safe (not registered,    recursive) ... FAIL
test_80_b2safe_local_for_multiple_directories_registered_recursive_01 (testB2SafeCmd.irodsb2safetest.IrodsB2safeIntegrationTests)
Test : replicate multiple directories locally using b2safe (    registered,    recursive, ror=pid, fio=pid, fixed_content=false) ... FAIL
test_80_b2safe_local_for_multiple_directories_registered_recursive_02 (testB2SafeCmd.irodsb2safetest.IrodsB2safeIntegrationTests)
Test : replicate multiple directories locally using b2safe (    registered,    recursive, ror=841/test, fio=None, fixed_content=false) ... FAIL
test_80_b2safe_local_for_multiple_directories_registered_recursive_03 (testB2SafeCmd.irodsb2safetest.IrodsB2safeIntegrationTests)
Test : replicate multiple directories locally using b2safe (    registered,    recursive, ror=None, fio=None, fixed_content=false) ... FAIL

======================================================================
FAIL: test_70_b2safe_local_for_one_file_not_registered_non_recursive (testB2SafeCmd.irodsb2safetest.IrodsB2safeIntegrationTests)
Test : replicate a single file locally using b2safe (not registered, non recursive)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/irodsmaster/B2SAFE-core/scripts/tests/testB2SafeCmd/irodsb2safetest.py", line 799, in test_70_b2safe_local_for_one_file_not_registered_non_recursive
    test_assert_array(self, test_values)
  File "/home/irodsmaster/B2SAFE-core/scripts/tests/testB2SafeCmd/irodsb2safetest.py", line 56, in test_assert_array
    self.assertEqual(result, expected, output_string)
AssertionError: The replication was NOT succesful

======================================================================
FAIL: test_70_b2safe_local_for_one_file_not_registered_recursive (testB2SafeCmd.irodsb2safetest.IrodsB2safeIntegrationTests)
Test : replicate a single file locally using b2safe (not registered,     recursive)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/irodsmaster/B2SAFE-core/scripts/tests/testB2SafeCmd/irodsb2safetest.py", line 833, in test_70_b2safe_local_for_one_file_not_registered_recursive
    test_assert_array(self, test_values)
  File "/home/irodsmaster/B2SAFE-core/scripts/tests/testB2SafeCmd/irodsb2safetest.py", line 56, in test_assert_array
    self.assertEqual(result, expected, output_string)
AssertionError: The replication was NOT succesful

======================================================================
FAIL: test_70_b2safe_local_for_one_file_registered_non_recursive_01 (testB2SafeCmd.irodsb2safetest.IrodsB2safeIntegrationTests)
Test : replicate a single file locally using b2safe (    registered, non recursive, ror=pid, fio=pid, fixed_content=false)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/irodsmaster/B2SAFE-core/scripts/tests/testB2SafeCmd/irodsb2safetest.py", line 906, in test_70_b2safe_local_for_one_file_registered_non_recursive_01
    test_assert_array(self, test_values)
  File "/home/irodsmaster/B2SAFE-core/scripts/tests/testB2SafeCmd/irodsb2safetest.py", line 56, in test_assert_array
    self.assertEqual(result, expected, output_string)
AssertionError: The replication was NOT succesful

======================================================================
FAIL: test_70_b2safe_local_for_one_file_registered_non_recursive_02 (testB2SafeCmd.irodsb2safetest.IrodsB2safeIntegrationTests)
Test : replicate a single file locally using b2safe (    registered, non recursive, ror=841/test, fio=None, fixed_content=false)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/irodsmaster/B2SAFE-core/scripts/tests/testB2SafeCmd/irodsb2safetest.py", line 979, in test_70_b2safe_local_for_one_file_registered_non_recursive_02
    test_assert_array(self, test_values)
  File "/home/irodsmaster/B2SAFE-core/scripts/tests/testB2SafeCmd/irodsb2safetest.py", line 56, in test_assert_array
    self.assertEqual(result, expected, output_string)
AssertionError: The replication was NOT succesful

======================================================================
FAIL: test_70_b2safe_local_for_one_file_registered_non_recursive_03 (testB2SafeCmd.irodsb2safetest.IrodsB2safeIntegrationTests)
Test : replicate a single file locally using b2safe (    registered, non recursive, ror=None, fio=None, fixed_content=false)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/irodsmaster/B2SAFE-core/scripts/tests/testB2SafeCmd/irodsb2safetest.py", line 1053, in test_70_b2safe_local_for_one_file_registered_non_recursive_03
    test_assert_array(self, test_values)
  File "/home/irodsmaster/B2SAFE-core/scripts/tests/testB2SafeCmd/irodsb2safetest.py", line 56, in test_assert_array
    self.assertEqual(result, expected, output_string)
AssertionError: The replication was NOT succesful

======================================================================
FAIL: test_70_b2safe_local_for_one_file_registered_recursive (testB2SafeCmd.irodsb2safetest.IrodsB2safeIntegrationTests)
Test : replicate a single file locally using b2safe (    registered,     recursive, ror=pid, fio=pid, fixed_content=false)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/irodsmaster/B2SAFE-core/scripts/tests/testB2SafeCmd/irodsb2safetest.py", line 1127, in test_70_b2safe_local_for_one_file_registered_recursive
    test_assert_array(self, test_values)
  File "/home/irodsmaster/B2SAFE-core/scripts/tests/testB2SafeCmd/irodsb2safetest.py", line 56, in test_assert_array
    self.assertEqual(result, expected, output_string)
AssertionError: The replication was NOT succesful

======================================================================
FAIL: test_75_b2safe_local_for_one_directory_not_registered_recursive (testB2SafeCmd.irodsb2safetest.IrodsB2safeIntegrationTests)
Test : replicate a single directory locally using b2safe (not registered,    recursive)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/irodsmaster/B2SAFE-core/scripts/tests/testB2SafeCmd/irodsb2safetest.py", line 1175, in test_75_b2safe_local_for_one_directory_not_registered_recursive
    test_assert_array(self, test_values)
  File "/home/irodsmaster/B2SAFE-core/scripts/tests/testB2SafeCmd/irodsb2safetest.py", line 56, in test_assert_array
    self.assertEqual(result, expected, output_string)
AssertionError: The replication was NOT succesful

======================================================================
FAIL: test_75_b2safe_local_for_one_directory_registered_recursive (testB2SafeCmd.irodsb2safetest.IrodsB2safeIntegrationTests)
Test : replicate a single directory locally using b2safe (    registered,    recursive, ror=pid, fio=pid, fixed_content=false)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/irodsmaster/B2SAFE-core/scripts/tests/testB2SafeCmd/irodsb2safetest.py", line 1276, in test_75_b2safe_local_for_one_directory_registered_recursive
    test_assert_array(self, test_values)
  File "/home/irodsmaster/B2SAFE-core/scripts/tests/testB2SafeCmd/irodsb2safetest.py", line 56, in test_assert_array
    self.assertEqual(result, expected, output_string)
AssertionError: The replication was NOT succesful

======================================================================
FAIL: test_80_b2safe_local_for_multiple_directories_not_registered_recursive (testB2SafeCmd.irodsb2safetest.IrodsB2safeIntegrationTests)
Test : replicate multiple directories locally using b2safe (not registered,    recursive)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/irodsmaster/B2SAFE-core/scripts/tests/testB2SafeCmd/irodsb2safetest.py", line 1338, in test_80_b2safe_local_for_multiple_directories_not_registered_recursive
    test_assert_array(self, test_values)
  File "/home/irodsmaster/B2SAFE-core/scripts/tests/testB2SafeCmd/irodsb2safetest.py", line 56, in test_assert_array
    self.assertEqual(result, expected, output_string)
AssertionError: The replication was NOT succesful

======================================================================
FAIL: test_80_b2safe_local_for_multiple_directories_registered_recursive_01 (testB2SafeCmd.irodsb2safetest.IrodsB2safeIntegrationTests)
Test : replicate multiple directories locally using b2safe (    registered,    recursive, ror=pid, fio=pid, fixed_content=false)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/irodsmaster/B2SAFE-core/scripts/tests/testB2SafeCmd/irodsb2safetest.py", line 1459, in test_80_b2safe_local_for_multiple_directories_registered_recursive_01
    test_assert_array(self, test_values)
  File "/home/irodsmaster/B2SAFE-core/scripts/tests/testB2SafeCmd/irodsb2safetest.py", line 56, in test_assert_array
    self.assertEqual(result, expected, output_string)
AssertionError: The replication was NOT succesful

======================================================================
FAIL: test_80_b2safe_local_for_multiple_directories_registered_recursive_02 (testB2SafeCmd.irodsb2safetest.IrodsB2safeIntegrationTests)
Test : replicate multiple directories locally using b2safe (    registered,    recursive, ror=841/test, fio=None, fixed_content=false)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/irodsmaster/B2SAFE-core/scripts/tests/testB2SafeCmd/irodsb2safetest.py", line 1580, in test_80_b2safe_local_for_multiple_directories_registered_recursive_02
    test_assert_array(self, test_values)
  File "/home/irodsmaster/B2SAFE-core/scripts/tests/testB2SafeCmd/irodsb2safetest.py", line 56, in test_assert_array
    self.assertEqual(result, expected, output_string)
AssertionError: The replication was NOT succesful

======================================================================
FAIL: test_80_b2safe_local_for_multiple_directories_registered_recursive_03 (testB2SafeCmd.irodsb2safetest.IrodsB2safeIntegrationTests)
Test : replicate multiple directories locally using b2safe (    registered,    recursive, ror=None, fio=None, fixed_content=false)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/irodsmaster/B2SAFE-core/scripts/tests/testB2SafeCmd/irodsb2safetest.py", line 1701, in test_80_b2safe_local_for_multiple_directories_registered_recursive_03
    test_assert_array(self, test_values)
  File "/home/irodsmaster/B2SAFE-core/scripts/tests/testB2SafeCmd/irodsb2safetest.py", line 56, in test_assert_array
    self.assertEqual(result, expected, output_string)
AssertionError: The replication was NOT succesful

----------------------------------------------------------------------
Ran 98 tests in 591.972s

FAILED (failures=12, skipped=27)

[cookie33]

Everywhere the replication is NOT successful it states. replication does work, but it fails with error messages:

Jun 27 12:26:01 pid:3738 NOTICE: writeLine: inString = DEBUG: [EUDATReplication] response = /tempZone/home/irsara/test_b2safe_data1.txt::/tempZone/home/irsara/test_b2safe_data2.txt::demoResc::registered=false+::recursive=true
Jun 27 12:26:01 pid:3738 NOTICE: writeLine: inString = true
Jun 27 12:26:01 pid:3738 remote addresses: 127.0.0.1, 145.100.4.16 ERROR: addMsParam: NULL msParamArray or label input
Jun 27 12:26:01 pid:3738 remote addresses: 127.0.0.1, 145.100.4.16 ERROR: addMsParam: NULL msParamArray or label input

This causes the return to be empty instead of Succes! or Failure!

12:25 sara-irods4.irods.surfsara.nl:/home/irsara/git/B2SAFE-core/scripts/tests 
irsara$ ./testB2SafeCmd.py -test b2safe
Test b2safe
test_10_irods_functions (testB2SafeCmd.irodsb2safetest.IrodsB2safeIntegrationTests)
Test : basic iRODS functions. ... ok
test_10_irods_put_file (testB2SafeCmd.irodsb2safetest.IrodsB2safeIntegrationTests)
Test : put a file in iRODS ... ok
test_70_b2safe_local_for_one_file_not_registered_recursive (testB2SafeCmd.irodsb2safetest.IrodsB2safeIntegrationTests)
Test : replicate a single file locally using b2safe (not registered,     recursive) ... Replica_result:******
21.T12996/7C1FA156-79F4-11E8-B26A-FA163E98A05A

FAIL

======================================================================
FAIL: test_70_b2safe_local_for_one_file_not_registered_recursive (testB2SafeCmd.irodsb2safetest.IrodsB2safeIntegrationTests)
Test : replicate a single file locally using b2safe (not registered,     recursive)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/irsara/git/B2SAFE-core/scripts/tests/testB2SafeCmd/irodsb2safetest.py", line 301, in test_70_b2safe_local_for_one_file_not_registered_recursive
    test_assert_array(self, test_values)
  File "/home/irsara/git/B2SAFE-core/scripts/tests/testB2SafeCmd/irodsb2safetest.py", line 56, in test_assert_array
    self.assertEqual(result, expected, output_string)
AssertionError: The replication was NOT succesful

----------------------------------------------------------------------
Ran 3 tests in 5.367s

FAILED (failures=1)

And the code which executes should always return Succes!

def replicate_irods_file(source, destination, registered, recursive):
    '''procedure to replicate a file with EUDAT rules in iRODS'''
    # create test rule to replicate file
    #irule_rule = "{*status = EUDATReplication(*source, *destination,*registered, *recursive, *response); writeLine('serverLog', str(*status)); if (*status) { writeLine('stdout', 'Success!'); } else { writeLine('stdout', 'Failed: *response'); }}"
    irule_rule = "{*status = EUDATReplication(*source, *destination,*registered, *recursive, *response); writeLine('serverLog', str(*status)); writeLine('stdout', 'Success!'); }"
    irule_input = '*source='+source+'%*destination='+destination+'%*registered='+registered+'+%*recursive='+recursive
    irule_output = 'ruleExecOut'
    command = ["irule", irule_rule, irule_input, irule_output]
    # execute test rule
    irule_result = subprocess_popen(command)
    replicate_result = irule_result[0]
    return replicate_result

Because the iRODS code is more strict somewhere a NULL or empty label is passed.

 int
   47 addMsParamToArray( msParamArray_t *msParamArray, const char *label,
   48                    const char *type, void *inOutStruct, bytesBuf_t *inpOutBuf, int replFlag ) {
   49     msParam_t **newParam;
   50     int len, newLen;
   51 
   52     if ( msParamArray == NULL || label == NULL ) {
   53         rodsLog( LOG_ERROR,
   54                  "addMsParam: NULL msParamArray or label input" );
   55         return SYS_INTERNAL_NULL_INPUT_ERR;
   56     }

[cookie33]

see also: irods/irods#3722
The real problem was already there in iRODS 4.2.2. But because msiexec did not work remote it was never solved.

[cookie33]

See also: irods/irods#3998
The remote service is broken.

[cookie33]

Waiting for fix in iRODS 4.2.4

[cookie33]

Fixed in iRODS 4.2.5. Available when 4.2.5 is released.