Particle refinement fails (no _boxes3d.txt file)

[JuhaHuiskonen]

I am getting this error in refinement and cannot figure out why. It occurs only in one project and all tilt series of this project. File csp/lamella11_ts_004_boxes3d.txt is not found. Any suggestions?

1	2024-02-08 15:45:26 [INFO] /opt/pyp/bin/run/pyp:3273 | Job 3186772_1 (v0.6.2) launching on c6174.mahti.csc.fi using 128 task(s) and 200 GB of RAM
2	2024-02-08 15:45:39 [INFO] /opt/pyp/bin/run/pyp:3200 | Filesystem                                  Size  Used Avail Use% Mounted on
3	2024-02-08 15:45:39 [INFO] /opt/pyp/bin/run/pyp:3200 | 10.141.0.14@o2ib:10.141.0.13@o2ib:/scratch   40T  4.7T   36T  12% /scratch/project_2009485/nextPYP/tmp
4	2024-02-08 15:45:39 [INFO] pyp/system/mpi.py:180 | First function is: load_files
5	2024-02-08 15:45:40 [INFO] pyp/system/mpi.py:180 | First function is: load_files
6	2024-02-08 15:45:40 [INFO] pyp/inout/utils/core.py:17 | Retrieving lamella11_ts_004.rec
7	2024-02-08 15:45:40 [INFO] pyp/inout/utils/core.py:17 | Retrieving lamella11_ts_004.pkl
8	2024-02-08 15:45:40 [INFO] pyp/inout/utils/core.py:17 | Retrieving lamella11_ts_004.mrc
9	2024-02-08 15:45:40 [INFO] pyp/inout/utils/core.py:17 | Retrieving lamella11_ts_004.webp
10	2024-02-08 15:45:40 [INFO] pyp/inout/utils/core.py:17 | Retrieving lamella11_ts_004_raw.webp
11	2024-02-08 15:45:40 [INFO] pyp/inout/utils/core.py:17 | Retrieving lamella11_ts_004_rec.webp
12	2024-02-08 15:45:40 [INFO] pyp/inout/utils/core.py:17 | Retrieving lamella11_ts_004_ali.webp
13	2024-02-08 15:45:40 [INFO] pyp/inout/utils/core.py:17 | Retrieving lamella11_ts_004_sides.webp
14	2024-02-08 15:45:48 [INFO] pyp:1795 | Executing coordinate extraction
15	2024-02-08 15:45:48 [INFO] pyp:2082 | Using image 20 [-0.] as zero tilt image
16	2024-02-08 15:45:48 [INFO] pyp:2093 | Views to exclude [1.0, 2.0]
17	2024-02-08 15:45:48 [INFO] pyp:2179 | The height of specimen in the tomogram is -2462.72 A
18	2024-02-08 15:45:48 [INFO] pyp:57 | Reading and converting coordinates took: 00h 00m 00s
19	2024-02-08 15:45:48 [INFO] /opt/pyp/bin/run/pyp:57 | Total time elapsed (csp_swarm): 00h 00m 09s
20	Traceback (most recent call last):
21	  File "/opt/pyp/bin/run/pyp", line 3914, in <module>
22	    csp_swarm(args.file, parameters, int(args.iter), args.skip, args.debug)
23	  File "/opt/pyp/src/pyp/utils/timer.py", line 78, in wrapper_timer
24	    return func(*args, **kwargs)
25	  File "/opt/pyp/bin/run/pyp", line 2376, in csp_swarm
26	    [allboxes, allparxs] = csp_extract_coordinates(
27	  File "/opt/pyp/src/pyp/utils/timer.py", line 78, in wrapper_timer
28	    return func(*args, **kwargs)
29	  File "/opt/pyp/src/pyp/inout/metadata/core.py", line 1890, in csp_extract_coordinates
30	    shutil.copy2("csp/{}_boxes3d.txt".format(filename), working_path)
31	  File "/usr/local/envs/pyp/lib/python3.8/shutil.py", line 435, in copy2
32	    copyfile(src, dst, follow_symlinks=follow_symlinks)
33	  File "/usr/local/envs/pyp/lib/python3.8/shutil.py", line 264, in copyfile
34	    with open(src, 'rb') as fsrc, open(dst, 'wb') as fdst:
35	FileNotFoundError: [Errno 2] No such file or directory: 'csp/lamella11_ts_004_boxes3d.txt'
36	2024-02-08 15:45:48 [ERROR] /opt/pyp/bin/run/pyp:3927 | PYP (cspswarm) failed
37	2024-02-08 15:45:52 [INFO] /opt/pyp/bin/run/pyp:3273 | Job 3186772_1 (v0.6.2) launching on c6174.mahti.csc.fi using 128 task(s) and 200 GB of RAM
38	2024-02-08 15:45:52 [INFO] pyp/refine/csp/particle_cspt.py:57 | Total time elapsed (csp_local_merge): 00h 00m 00s
39	Traceback (most recent call last):
40	  File "/opt/pyp/bin/run/pyp", line 3992, in <module>
41	    particle_cspt.merge_movie_files_in_job_arr(
42	  File "/opt/pyp/src/pyp/utils/timer.py", line 78, in wrapper_timer
43	    return func(*args, **kwargs)
44	  File "/opt/pyp/src/pyp/refine/csp/particle_cspt.py", line 380, in merge_movie_files_in_job_arr
45	    with open(movie_file) as f:
46	FileNotFoundError: [Errno 2] No such file or directory: 'stacks.txt'
47	2024-02-08 15:45:52 [ERROR] /opt/pyp/bin/run/pyp:4012 | PYP (csp_local_merge) failed
48	2024-02-08 15:45:53 [INFO] /opt/pyp/bin/run/pyp:4018 | Deleted temporary files from /scratch/project_2009485/nextPYP/tmp/huiskone/3186772_1

[tofushan]

Hi,

It means this specific tilt-series do not have any valid particles. It can be no particle present in this tilt-series or no particles present in your initial parameter file (.txt). If this is latter, check if the name of your particles present in the last column of the file (usually starts with the name of tilt-series followed by the ID of particles, i.e., TS_01_spk0000.rec means the first particle in tilt-series TS_01).

I hope this helps.

[yezhou867]

If there is a tomogram (tilt series) with no particles picked, it will be removed automatically during the end of preprocessing. In the merge log of preprocessing, you should see something like this indicating certain images are removed because no particles exist.