Tailseeker2 error in calculate-pasignals

[roelj]

Hi there!

Thank you for this pipeline. Unfortunately I am running into an issue:

Traceback (most recent call last):
  File "/gnu/store/9z98cvjm7p7z21xdid1ryydxy5vyz6wr-python-3.7.0/lib/python3.7/concurrent/futures/process.py", line 232, in _process_worker
    r = call_item.fn(*call_item.args, **call_item.kwargs)
  File "/gnu/store/hbmh3jzpwm2rxf9sldzlv335h6hm7b6l-tailseeker-2.0/share/tailseeker/scripts/calculate-pasignals.py", line 92, in run_single_job
    adjsignal *= tstretchadj[row.tile][:len(adjsignal)]
IndexError: invalid index to scalar variable.

Do you know what the problem could be here, or whether and where I misconfigured the pipeline?

The full traceback is this:

rule calculate_pasignals_v2:
    input: signalproc/colormatrix.pickle, sequences/Fucci-100-M-TAIL-seq.sqi.gz, sequences/Fucci-100-M-TAIL-seq.sqi.gz.tbi, signalproc/signal-scaling.phix-ref.pickle, signalproc/signal-scaling-Fucci-100-M-TAIL-seq.stabilizer.pickle
    output: scores/Fucci-100-M-TAIL-seq.pa2score.gz, stats/Fucci-100-M-TAIL-seq.pascore-calculation.csv
    jobid: 18
    wildcards: sample=Fucci-100-M-TAIL-seq
    threads: 16

Job counts:
	count	jobs
	1	calculate_pasignals_v2
	1
Traceback (most recent call last):
  File "/gnu/store/hbmh3jzpwm2rxf9sldzlv335h6hm7b6l-tailseeker-2.0/share/tailseeker/scripts/calculate-pasignals.py", line 92, in run_single_job
    adjsignal *= tstretchadj[row.tile][:len(adjsignal)]
IndexError: invalid index to scalar variable.
concurrent.futures.process._RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/gnu/store/9z98cvjm7p7z21xdid1ryydxy5vyz6wr-python-3.7.0/lib/python3.7/concurrent/futures/process.py", line 232, in _process_worker
    r = call_item.fn(*call_item.args, **call_item.kwargs)
  File "/gnu/store/hbmh3jzpwm2rxf9sldzlv335h6hm7b6l-tailseeker-2.0/share/tailseeker/scripts/calculate-pasignals.py", line 92, in run_single_job
    adjsignal *= tstretchadj[row.tile][:len(adjsignal)]
IndexError: invalid index to scalar variable.
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/gnu/store/hbmh3jzpwm2rxf9sldzlv335h6hm7b6l-tailseeker-2.0/share/tailseeker/scripts/calculate-pasignals.py", line 127, in run_jobs
    res = j.result()
  File "/gnu/store/9z98cvjm7p7z21xdid1ryydxy5vyz6wr-python-3.7.0/lib/python3.7/concurrent/futures/_base.py", line 425, in result
    return self.__get_result()
  File "/gnu/store/9z98cvjm7p7z21xdid1ryydxy5vyz6wr-python-3.7.0/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
IndexError: invalid index to scalar variable.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/gnu/store/hbmh3jzpwm2rxf9sldzlv335h6hm7b6l-tailseeker-2.0/share/tailseeker/scripts/calculate-pasignals.py", line 177, in <module>
    run_jobs(options)
  File "/gnu/store/hbmh3jzpwm2rxf9sldzlv335h6hm7b6l-tailseeker-2.0/share/tailseeker/scripts/calculate-pasignals.py", line 136, in run_jobs
    statstbl.to_csv(options.output_stats, index=False)
  File "/gnu/store/hbmh3jzpwm2rxf9sldzlv335h6hm7b6l-tailseeker-2.0/lib/python3.7/site-packages/tailseeker/fileutils.py", line 283, in __exit__
    shutil.rmtree(self.path)
  File "/gnu/store/9z98cvjm7p7z21xdid1ryydxy5vyz6wr-python-3.7.0/lib/python3.7/shutil.py", line 489, in rmtree
    onerror(os.rmdir, path, sys.exc_info())
  File "/gnu/store/9z98cvjm7p7z21xdid1ryydxy5vyz6wr-python-3.7.0/lib/python3.7/shutil.py", line 487, in rmtree
    os.rmdir(path)
OSError: [Errno 39] Directory not empty: './tmpe6fihlyi'
[Wed Feb 20 14:11:58 2019]
Error in rule calculate_pasignals_v2:
    jobid: 0
    output: scores/Fucci-100-M-TAIL-seq.pa2score.gz, stats/Fucci-100-M-TAIL-seq.pascore-calculation.csv

RuleException:
CalledProcessError in line 433 of /gnu/store/hbmh3jzpwm2rxf9sldzlv335h6hm7b6l-tailseeker-2.0/share/tailseeker/tailseeker/main.py:
Command ' set -e; set -o pipefail; BGZIP_CMD="/gnu/store/1d010fyd8aasa39gwkqhzwqcriji7rw4-htslib-1.9/bin/bgzip" TABIX_CMD="/gnu/store/1d010fyd8aasa39gwkqhzwqcriji7rw4-htslib-1.9/bin/tabix" TAILSEQ_SCRATCH_DIR="/hpc/cog_bioinf/cuppen/project_data/Roel_tailseeker_test/scratch" ;  /gnu/store/9z98cvjm7p7z21xdid1ryydxy5vyz6wr-python-3.7.0/bin/python3 /gnu/store/hbmh3jzpwm2rxf9sldzlv335h6hm7b6l-tailseeker-2.0/share/tailseeker/scripts/calculate-pasignals.py --parallel 16                 --output-stats stats/Fucci-100-M-TAIL-seq.pascore-calculation.csv                 --scaling-params signalproc/signal-scaling-Fucci-100-M-TAIL-seq.stabilizer.pickle sequences/Fucci-100-M-TAIL-seq.sqi.gz                 > scores/Fucci-100-M-TAIL-seq.pa2score.gz ' returned non-zero exit status 1.
  File "/gnu/store/hbmh3jzpwm2rxf9sldzlv335h6hm7b6l-tailseeker-2.0/share/tailseeker/tailseeker/main.py", line 433, in __rule_calculate_pasignals_v2
  File "/gnu/store/ibxxisi81v1v3y5xc3axl4x4rx6jpzw8-python-3.7.0/lib/python3.7/concurrent/futures/thread.py", line 57, in run
Exiting because a job execution failed. Look above for error message

Thank you for your time and efforts!

[byee4]

I had maybe a similar problem which was caused by some upstream step not producing an output, which itself was caused by a misconfiguration of one of my conf files. My issue happened to be that my delimiter sequence was incorrect, causing too many reads to be filtered out prior to the calculate_pasignal step.