No such file or directory for MotionCorr3 averages

[sam-mahdi]

Hello,

I came across an issue when running MotionCorr3, where it seems it wasn't able to generate an average? I had no pre-processing with unblur with this dataset.

2026-03-31 17:16:10 [INFO] pyp/pyp_main.py:4443 \| Job 240_1 running nextPYP v0.7.3 on chiem1.nhlbi.nih.gov using 5 task(s) and 5 GB of RAM
--
4 | 2026-03-31 17:16:10 [INFO] pyp/pyp_main.py:4475 \| Created temporary folder /scratch/nextpyp/nhlbi_nextpyp/240_1
5 | 2026-03-31 17:16:10 [INFO] pyp/pyp_main.py:4053 \| Filesystem                                             Size  Used Avail Use% Mounted on
6 | 2026-03-31 17:16:10 [INFO] pyp/pyp_main.py:4053 \| /dev/mapper/luks-36e099c1-abe3-4075-9683-5da7945a4574   15T   13T  2.3T  85% /scratch/nextpyp
7 | 2026-03-31 17:16:10 [INFO] pyp/pyp_main.py:1715 \| Using temporary folder /scratch/nextpyp/nhlbi_nextpyp/240_1/tomo103
8 | 2026-03-31 17:16:11 [INFO] pyp/utils/timer.py:75 \| Retrieving files took: 1.2s
9 | 2026-03-31 17:16:14 [INFO] pyp/system/mpi.py:178 \| Running 61 function(s) (copy2)
10 | Progress: 100%\|##########\| 61/61 [04:58<00:00,  4.89s/it]
11 | 2026-03-31 17:21:12 [INFO] pyp/system/mpi.py:181 \| 61 functions(s) finished
12 | 2026-03-31 17:21:30 [INFO] pyp/preprocess/core.py:623 \| Processing movie frames using: motioncor
13 | Progress:   0%\|          \| 0/61 [00:09<?, ?it/s]
14 | 2026-03-31 17:21:45 [INFO] pyp/utils/timer.py:75 \| Total time in tomo_swarm 5m 35.6s seconds
15 | 2026-03-31 17:21:45 [ERROR] pyp/pyp_main.py:4648 \| nextPYP (tomoswarm) failed
16 | 2026-03-31 17:21:46 [ERROR] pyp/system/local_run.py:70 \| sh: 0: getcwd() failed: No such file or directory
17 |  
18 | 2026-03-31 17:21:46 [INFO] pyp/pyp_main.py:4053 \| Filesystem                                             Size  Used Avail Use% Mounted on
19 | 2026-03-31 17:21:46 [INFO] pyp/pyp_main.py:4053 \| /dev/mapper/luks-36e099c1-abe3-4075-9683-5da7945a4574   15T   13T  2.3T  85% /scratch/nextpyp
20 | 2026-03-31 17:21:46 [INFO] pyp/pyp_main.py:6413 \| Deleted temporary folder /scratch/nextpyp/nhlbi_nextpyp/240_1
21 | Traceback (most recent call last):
22 | File "/opt/conda/envs/pyp/lib/python3.8/shutil.py", line 791, in move
23 | os.rename(src, real_dst)
24 | FileNotFoundError: [Errno 2] No such file or directory: 'tomo103_178_060_-60.0_DWS.mrc' -> '../tomo103_178_060_-60.0.avg'
25 |  
26 | During handling of the above exception, another exception occurred:
27 |  
28 | Traceback (most recent call last):
29 | File "/opt/pyp/bin/run/pyp", line 4643, in <module>
30 | tomo_swarm(args.path, args.file, args.debug, args.keep, args.skip)
31 | File "/opt/pyp/src/pyp/utils/timer.py", line 97, in wrapper_timer
32 | return func(*args, **kwargs)
33 | File "/opt/pyp/bin/run/pyp", line 1833, in tomo_swarm
34 | ] = preprocess.read_tilt_series(
35 | File "/opt/pyp/src/pyp/preprocess/core.py", line 629, in read_tilt_series
36 | align.align_movie_frames( parameters, frame_name, file_format, isfirst)
37 | File "/opt/pyp/src/pyp/align/core.py", line 4447, in align_movie_frames
38 | shutil.move( name + "_DWS.mrc", f"../{aligned_average}")
39 | File "/opt/conda/envs/pyp/lib/python3.8/shutil.py", line 811, in move
40 | copy_function(src, real_dst)
41 | File "/opt/conda/envs/pyp/lib/python3.8/shutil.py", line 435, in copy2
42 | copyfile(src, dst, follow_symlinks=follow_symlinks)
43 | File "/opt/conda/envs/pyp/lib/python3.8/shutil.py", line 264, in copyfile
44 | with open(src, 'rb') as fsrc, open(dst, 'wb') as fdst:
45 | FileNotFoundError: [Errno 2] No such file or directory: 'tomo103_178_060_-60.0_DWS.mrc'

For context this is my settings, if I keep everything the same and change to unblur, I have no errors

movie_weights | : | true
-- | -- | --
data_path_mdoc | : | /nextpyp_particles_test/IMOD/mdoc_files/*
tomo_ali_patches_x | : | 3
tomo_ali_patches_y | : | 3
ctf_max_res | : | 5
movie_motioncor_corr_interp | : | true
slurm_memory_per_task | : | 1
tomo_rec_thickness | : | 3712
slurm_tasks | : | 5
tomo_rec_binning | : | 4
movie_ali | : | motioncor
slurm_launch_memory_per_task | : | 1
scope_dose_rate | : | 2.1
movie_force_integer | : | true
movie_motioncor_bin | : | 2
scope_wgh | : | 0.1
micromon_block | : | tomo-pure-preprocessing
scope_pixel | : | 1.328
scope_voltage | : | 300
scope_tilt_axis | : | 88.6
data_path | : | /nextpyp_particles_test/IMOD/frames/*.tif
data_parent | : | /projects/test-289lyu/tomo-rawdata-ga3aWFcyMEToMLnk
data_mode | : | tomo
movie_mdoc | : | true
movie_magcorr | : | true
movie_motioncor_in_frame_motion | : | true
movie_motioncor_patch_x | : | 3
movie_motioncor_patch_y | : | 3
movie_motioncor_sumrange_max | : | 0
tomo_ali_method | : | aretomo3
tomo_ali_pixels_trim_x | : | 50
tomo_ali_pixels_trim_y | : | 50
tomo_rec_2d_filtering_method | : | doseweighting

</div>

[abartesaghi]

This looks like an issue related to the dose weighting and exposure limits used by motioncor3. Apparently, the program only generates the third *_DWS.mrc file when movie_motioncor_sumrange_max != 0. In your case, since movie_motioncor_sumrange_max = 0, no DWS file is produced by motionmor3. As a result, nextPYP cannot find the file, which leads to the error.

Possible solutions:

1. If your intention is not to impose an upper limit during dose weighting, avoid setting movie_motioncor_sumrange_max = 0. Instead, set it to the total dose, for example: movie_motioncor_sumrange_max = 2.1 × number_of_frames.

2. If your goal is to exclude the first ~3 e⁻/Ų, you can set both movie_motioncor_sumrange_min = 0 and movie_motioncor_sumrange_max = 0, and instead discard the initial frame(s) by setting First frames to 1 (movie_first = 1) or 2 (movie_first = 2).

Let us know if this solves the problem.

[sam-mahdi]

I have been able to eliminate the error by setting the max to the total dose, however the tomograms look like just noise when I use motioncorr3. I have tried to play around with the settings but they don't see to change anything, even low frequency information is gone. Do not have any problems when using unblur.

[abartesaghi]

That's strange, I wonder if there would be anything in the MotionCor3 logs that could point to the problem.