Threshold Scan fails the analysis

[mexanick]

Brief summary of issue

After doing the threshold scan the analysis fails with:

  File "/usr/lib/python2.7/site-packages/gempython/gemplotting/utils/threshAlgos.py", line 214, in anaUltraThreshold
    detName.push_back(rp.tree2array(thrTree, branches = [ 'detName' ] )[0][0][0])
IndexError: index 0 is out of bounds for axis 0 with size 0

Types of issue

• Bug report (report an issue with the code)
• Feature request (request for change which adds functionality)

Steps to Reproduce (for bugs)

anaUltraThreshold.py /data/bigdisk/GEM-Data-Taking/GE11_QC8//GE11-X-S-FRASCATI-0006/threshold/channel//2020.03.12.10.55/ThresholdScanData.root -c short
Initializing Histograms
ge11
Filling Histograms
Traceback (most recent call last):
  File "/opt/cmsgemos/bin/anaUltraThreshold.py", line 39, in <module>
    args.func(args,args.infilename,args.GEBtype,filePath,args.fileScurveFitTree)
  File "/usr/lib/python2.7/site-packages/gempython/gemplotting/utils/threshAlgos.py", line 214, in anaUltraThreshold
    detName.push_back(rp.tree2array(thrTree, branches = [ 'detName' ] )[0][0][0])
IndexError: index 0 is out of bounds for axis 0 with size 0

Possible Solution (for bugs)

Context (for feature requests)

Your Environment

• Version used:

gempython_gemplotting
Arch        : noarch
Version     : 1.5.11

gempython_vfatqc
Arch        : x86_64
Version     : 2.7.11

• Shell used:

[jsturdy]

This is probably related to the version of numpy or root_numpy not being well defined in the dependencies and either a local install or system install not matching the API @bregnery
I put in the extras repo,
numpy-1.16.5-1
root_numpy-4.7.3-1.gemos.numpy1165.root618.el7.x86_64.rpm
root_numpy-4.8.0-1.gemos.numpy1165.root618.el7.x86_64.rpm
so that a system install should work with what was previously mentioned as the target version of numpy (along with requiring ROOT 6.18)
We should push these out system wide, and ensure that bugs are able to be reported against those specific versions

[mexanick]

but we have:

[gemuser@gem904qc7daq ~]$ root --version
ROOT Version: 6.16/00
Built for linuxx8664gcc on Jan 23 2019, 09:06:13
From tags/v6-16-00@v6-16-00

shall it be updated too?

[jsturdy]

but we have:

[gemuser@gem904qc7daq ~]$ root --version
ROOT Version: 6.16/00
Built for linuxx8664gcc on Jan 23 2019, 09:06:13
From tags/v6-16-00@v6-16-00

shall it be updated too?

Yes, unless we know of some incompatibility somewhere with that version (but in 904 there are 4 versions of ROOT on various machines, so it would be good to regularize, and as @lpetre-ulb mentioned elsewhere, 6.18 provides some fixes that would be globally useful)

[mexanick]

the extras repo on the gem904qc7daq points to
http://cmsgemdaq.web.cern.ch/cmsgemdaq/sw/repos/centos7_x86_64/extras/RPMS/
This repo has not been updated since 2019 and doesn't have the packages you've mentioned above.
The NAS repo has

drwxrwsr-x. 3 daqpro daqpro 4.0K Nov 27 11:37 ..
-rw-r--r--. 1 sturdy daqpro 435K Nov 28 12:14 root_numpy-4.8.0-1.numpy1165.root614.gcc485.el7.x86_64.rpm
-rw-r--r--. 1 sturdy daqpro 435K Nov 28 12:14 root_numpy-4.8.0-1.numpy1165.root618.gcc485.el7.x86_64.rpm
-rw-r--r--. 1 sturdy daqpro 435K Dec  2 16:03 root_numpy-4.8.0-1.numpy1165.root612.gcc485.el7.x86_64.rpm
-rw-r--r--. 1 sturdy daqpro 4.3M Dec  2 16:04 numpy-1.16.5-1.el7.x86_64.rpm
-rw-r--r--. 1 sturdy daqpro 436K Dec  2 16:09 root_numpy-4.8.0-1.numpy1165.root616.gcc485.el7.x86_64.rpm
-rw-r--r--. 1 sturdy daqpro 4.3M Dec  3 15:05 numpy-1.16.5-2.gemos.el7.x86_64.rpm
drwxr-sr-x. 2 root   daqpro 4.0K Mar 12 13:01 repodata
drwxr-sr-x. 3 sturdy daqpro 4.0K Mar 12 13:01 .

Which repo should be used?

[mexanick]

executed suggested updates, still getting the same error

[AndrewLevin]

did you check that the input data is good (at least not empty)?

[mexanick]

I was assured that the data is correct by QC7 crew.
However upon checking the log, I see:

 cat /data/bigdisk/GEM-Data-Taking/GE11_QC8//GE11-X-S-FRASCATI-0006/threshold/channel//2020.03.12.10.55/scanLog.log
2020.03.12.10.55
Open pickled address table if available  /opt/cmsgemos/etc/maps//amc_address_table_top.pickle...
Initializing AMC gem-shelf02-amc10
opened connection
Setting CFG_SEL_COMP_MODE to 0x1 (ARM Mode)
Do not force ZCC output
attempting to configure TTC
TTC configured successfully
Channel #0
Caught an error: memsvc error: Bus error accessing 0x6540421c
An exception occurred RPC response was non-zero, threshold scan for channel 0 failed

Apparently two actions are required:

1. Graceful handling of such broken files here, but the analysis code
2. Proper reporting of the errors - I believe they missed the error as in order to see it one needs to look into the log file manually

[lpetre-ulb]

1. Graceful handling of such broken files here, but the analysis code

Indeed, but should it be done in the legacy release? Or only in the updated analysis tools (if they are maintained)?

2. Proper reporting of the errors - I believe they missed the error as in order to see it one needs to look into the log file manually

This is known defect with a well known workaround procedure. Maybe not so well known in the end... Not sure if it is worth being fixed in the legacy release, but it must certainly be documented.