vminmon jobs can fail silently

[DavidHuber-NOAA]

What is wrong?

As reported here, when the minmon jobs call the ush/ Perl scripts, it seems that Perl does not return a non-0 status if there is an internal failure.

What should have happened?

These scripts should be hardened to make sure that minimization monitoring does not silently fail.

What machines are impacted?

All or N/A

What global-workflow hash are you using?

e95f278

Steps to reproduce

1. Modify
   

   global-workflow/ush/minmon_xtrct_reduct.pl

   Line 80 in e464779

   system("cp -f $outfile $newfile");

   
   to read
   system("cpfs -f $outfile $newfile");
   (the above is an illegal command)
2. Run a C96_atm3DVar test case through the gdas_vminmon job.

Additional information

No response

Do you have a proposed solution?

No response

[DavidHuber-NOAA]

FYI @EdwardSafford-NOAA